Ballroom C Friday Mar. 06 - SCaLE 23x

This talk at Sunset Con 2 features Farzan Karimi, who explores the importance of collaboration in cybersecurity efforts. He highlights the challenges faced due to silos within organizations, such as conflicts between security teams and product development teams. Farzan shares his experiences on how effective collaboration can lead to successful outcomes against cyber threats, drawing on stories from his past, including his time at Google and EA. He discusses the common misconceptions about security responsibilities and emphasizes the critical need for open communication. Ultimately, Farzan argues that the true vulnerability in organizations often lies not in technical flaws but in human isolation, and he advocates for building a culture of trust and teamwork to improve security defenses.

Okay. We're going to start in about a few minutes. We're just waiting for them to fix the uh pre projector and then we'll get started. Thank you. All right. Test. >> All right. Thank you everyone for joining us today and welcome to Sunset Con 2. Uh we're happy to have you here. I know it's early, but we definitely have a full day um with a lot of

good talks. If you were here yesterday uh and attended the workshops, they were also really great. So, thank you for the ones who attended. Uh my name is Marco Palasios. I'm the president co-founder of Pacific Hackers and also one one of the organizers for uh Sunset Con. Uh before we get started, uh you know, at Pacific Hackers, we have a rule and we have a no rule.

So, we don't have egos. We don't have uh you know, we respect everyone. So, please uh do the same. Uh I know there's so much going on right now in the world. Uh unfortunately, not so positive. So, uh you know, I know we have freedom of speech, but just keep things for yourself and uh you know, I think we can do a better job than what's going

on around the world. Um anyways, uh Pacific Hackers, if you have not um gone through our booth, uh we're a 501c3 uh nonprofit organization here in uh California. We're headquartered in Silicon Valley and uh we're basically placing people into the the field. But while we're doing that, we're also um fixing some of the issues in the field, diversity being one of them. We want more women in

the field, more people of color, the education, right? uh we know there's so many programs uh universities uh you know teaching cyber security realistically and that honestly is the true uh there's so much politics in in education that you know sometimes the curriculum it's outdated and they're not really teaching the right stuff right so with that being said you know students have to once they graduate and

are in deaf they have to come back and you know join uh places like Pacific Hackers uh subsequently we're fixing the HR you know, the placement uh when you join Pacific Hackers, you know, we don't have a membership program, but just by joining our meetups, you're part of a community, part of a family. And you know, that's how we're being able to place people into the jobs

because you know, you hire from your family, right? Because you know that person. So when you come in and even right here, you could potentially be potentially be looking at your uh colleague, you know, your hiring manager, who knows? Um, so again, uh, if you we're trying to expand to Southcal, so if you're in the Southcal area, we'll be here. I don't know how soon, but there's

a promise that we're going to be here. Um, so, you know, look into that. Um, so we have diff different type of merch. Uh, definitely check us out. That's how we're able to sustain the, uh, the nonprofit. And again, uh, we could not be here without, you know, Scale. So again, thank you Scale for for the opportunity. And, um, you know, uh just want to rush things

out because we have a great presenter and a lot of great presenters today. Um this how you can follow us. Uh there's uh business cards on at the table. So definitely um um grab one of those. Uh please if you take pictures, uh share them. Here is a QR code where you can upload your pictures and we will be happy to to have them. So uh thank

you. And with that, um, I'll pass it on to our keynote speaker. Uh, you know, I I met him, uh, about a month ago and I saw one of his talks and I was like, "Yeah, this guy is definitely what I'm looking for for uh, Sunset Con." And happens to be that he's in the area, so he didn't have to travel far. So, uh, with that, I'll

pass it on to Faran. Uh, thank you so much for joining us today. Uh, taking the, uh, the trip. Um, how long it took you? 45 hour. an hour and 20, but here you go. Well, still uh but thank you so much for joining us and um yeah, go ahead. >> Great. Thank you so much, Marco. Thank you all. Just bear with me a sec as I

plug the HDMI in here. All right, sweet. Um, let me know too if my audio cut is a little too loud or if the volume's good. So far, we're good. Cool. Perfect. Well, thank you so much again. This is uh awesome to be here part of Sunset Con. My first time at scale and Sunsetcon and it's a really treat to just be invited and meet you all.

U my name is Farzan Karimi. Uh I'm absolutely thrilled again to be here. Uh just because this is my first time I want to sort of map out the room. So how many here are primarily for scale more like DevOps infrastructure engineering? Okay. How many software developers? All right. A little bit more. What about just security focused people? Sweet. Okay, so we have builders, breakers, and people

who have to clean the mess up from what I'm reading, right? Okay, awesome. Um, I gave this keynote uh at a conference called Shield Con a few months ago uh out in Norway and it's kind of a an interesting contrast to go from the coldest place on Earth to here in like sunniest uh sunny California. So, it's been really awesome. And I have to say one of

the things that I see kind of just uh transcend between both locations is just you can go into a community where you don't know anybody and then immediately feel welcome. And that's what I felt out in Norway. Um and just again just brilliant engineers, really passionate people that want to collaborate and welcome you into their community. And I'm already getting the sense of that here at scale.

So again, thank you for having me. So I'm going to start with a framing premise for this session and it's hackers collaborate right they share tools they share knowledge and depending on their level of sophistication they may share that access to their AB and C teams they may sell off that access to external parties defenders we're okay you know and as effective as we are at the

individual or team level we're often victims of these organizational silos that trap us into not being able to collaborate well with each other. So, think of security versus uh software teams, product versus enterprise, blue team versus red team, right? These are interesting dynamics. And if you're on a enterprise team and you touch the scope of a product team, watch out. So I was once part of a

red team where I was running a an engagement and adversary simulation exercise where I found a way to evade detection and the methodology I used was just disabling EDR, right? Disabling the sensor. My counterparts on the blue team weren't really happy with that methodology because their expectation was that I would keep EDR on so that they could see how I'm evading detections with that visibility. My position

was that well an attacker if they can turn it off they'll turn it off. So it didn't that engagement didn't end well and it led to me being called a cheater on one trick pony and somebody called me defender suspender which in retrospect was pretty clever. Um but it actually kind of hurt in that moment because it made me feel more isolated from the blue team. Um,

and so I'm here to just kind of share a little bit of these experiences through a couple stories. Uh, where collaboration when done right can really uh, uh, transcend the attacker and lead to arrests, which I'll talk about, but also when collaboration uh, kind of fell apart and didn't happen and led to the attacker getting the upper hand. So just quick on me, I'm Farzan Karimi. I'm

the deputy CISO as of this week. I'm the deputy CISO of MADNA. It's a biotech company based in Boston. Thank you. Um I was kind of like egging for a little bit of clapping. Thanks. Uh uh if you're not uh familiar with it, we focus on therapeutic development for uh cancer treatment and is going to be hitting some big milestones in 2027, which I'm excited by. Uh

it's a Bostonbased company. I am Southern California based, thank God. Uh have you seen the weather out in Boston and the East Coast the past two months? No thanks. Uh I've led offensive security teams for the better part of 15 years, 20 years in total. I started as a software engineer, then uh moved into offsac. I managed the red team at Google, specifically the Android red team

for a couple years. Then man or before that managed the EA red team, uh where I was the founding member and right now uh grew into more of the security response function with where I'm at now. I spoke at Black Hat and Defcon with my Google rep team and just this past summer I spoke at Defcon at an individual capacity as a security researcher. Awesome experience. So

again, just uh gracious for the invite to speak to you all. So this is where we're headed and I'll start with this premise that the real zero day in companies isn't necessarily a technical flaw. It's isolation. Now I'm not talking about network isolation. I'm talking about human isolation. So, think about these social pressures that we deal with on a day-to-day basis, right? Whether there's fear of collaborating,

like fear of looking stupid, right? The hero mentality, too. You may want to just work your hardest to to knock something out on your own without including others that and that's generally rewarded, right? So, this is this dynamic really kind of um creates problems and I'm going to be sharing stories, some that had successful collaboration, some not so much. Um, and it's not necessarily about what happened

necessarily with an exploit. It's the human behavior after the exploit. These 10 situations where they escalated or deescalated based on how the conversations went. Um, so these are a couple of the stories. We're going to get to it. And again, uh the whole point is we have a shared goal here to unite ourselves together in order to defeat the attacker. So I'm going to start um actually

uh I mentioned like a few failure stories. I'll mention a really recent failure story that came out of Defcon last summer. So again, I presented on this new web attack methodology called RRE. And 17 hours before I went on stage, something happened. I got a call from the legal department from one of the companies that was impacted by a vulnerability and they sent me an email, a

LinkedIn message, a text and something else. I forget the fourth one. Uh, and they said, "Farison, we heard through a journalist that you're going public with this vulnerability details. And but based on our understanding, you are in violation of responsible disclosure policy. And if you go on stage, there will be legal implications." So this is 17 hours before my talk that I prepped months and months for.

So I'm starting to freak out internally and I get on the call and it was just really just a big miscommunication. I did responsibly disclose the issue but they were expecting it through their hacker one channel. I sent it via email. They didn't check email or it was an unmonitored mailbox. But the the whole key point of that story is that I thought I was in the

Right? And ethically I thought I took all the right steps but the optics were different you know and it doesn't matter whether you do the right thing if the optics are different you find yourself in a very difficult conversation right and you have to find ways to deescalate in those tense situations. So, this is actually I'm going to start with this kind of a successful collaboration story

from a time when I was at EA Electronic Arts, a big video game company, uh, that was just sold for a whopping 55 billion last year. Um, and if you're a former EA employee like me, you might be thinking, should have kept some of that stock. if for those who don't know, uh, one of the biggest games at EA is FIFA. And FIFA has one of the

largest virtual currency platforms in the entire gaming industry. So with FIFA Ultimate Teams, you can leverage FIFA coins, you can buy FIFA coins to build out your dream team. You can buy merch, you can do all sorts of things, which makes it a very attractive virtual currency in the gaming market. It also makes it very attractive for attackers to target, believe it or not. So there's organized

crime units, uh, AP, etc. that had targeted FIFAcoin, which is kind of funny to say out loud, but it is a problem in the gaming industry. So, one hacker that was based in Europe um found a vulnerability. And by the way, as as I speak through this, um this is all public information. Uh this has been discussed in various media articles and I'm just kind of giving

my on the ground perspective on it being involved in this case. So this hacker found a way to target some external facing API APIs for FIFA and generate FIFA coins. Um just imagine for example like multiplayer matchmaking you know if you can spoof being able to win and play matches you could generate some FIFA coins. So automating things like that. What this attacker did was still steal

around $324,000 worth of FIFA coins. Right. Now, it gets more interesting because the next thing he did was distribute those 25,000 uh distribute those $324,000 worth of FIFA coin across 25,000 accounts, stranger accounts. And you may be thinking, yeah, this guy's not too bad, right? Like he's kind of like a digital Robin Hood just giving FIFA coin back to the people. um you know a poking at

big bad EA until you do a little bit of analysis and realize that oh some of his accounts were in that set of 25,000 so he's trying to offis you skcate where the money's going right how did we come to know about it well I'll tell you on the next slide but uh what makes this more interesting and why I'm even bringing this story is just because

all the cards were stacked against us right from a defensive perspective with this this a threat actor was based in a different country, they were stealing all this money and offiscating where the money is flowing. So, it makes it really hard for us to pinpoint who the attacker is, but yet good collaboration across multiple teams led to this person's arrest. So, how did that happen? So, the

attacker started just kind of a DFD here. The attacker exploited FIFA coins, as I mentioned, uh took out worth. The blue team sees that something's a miss. that amount of money should be should have a detection associated with it. They start looking at the logs and they enumerate all the APIs that the threat actor targeted, right? And this is where, you know, blue teams may continue to

do the forensics investigation on their own, which is perfectly all right. Um, but they did something different. They looped in my red team at this point and they asked our red team, hey, if we give you all these logs, can you replay this attack as a form of evidence that this is exactly what the threat actor did? And we thought that that was an awesome opportunity to

be involved in an incident in a way that brings down a criminal. So, we did that. Guess what? We mapped out the exact API path this actor took. It lined up with the logs. We took that as evidence and provided that to our legal council who then provided that as a courtroom narrative to give to the judge who then thought, hey, this person should go to jail,

right? But again, this person's based in Italy at this time. So, how would you arrest somebody that's in another country uh that's hacking your your company's assets and services? Does anybody have an idea? >> Yes. Yes. Yes. Very good. Yeah. Invitation to do something. Right. So I you know this is part where I wasn't involved but um you know so so exactly two observations here to follow

on your your answer which is very good is that in the US if you steal around a certain threshold of money you get on the radar of the FBI because it's a federal crime at that point then you're on FBI watch list. Observation two is you know again to arrest somebody it's much easier if they're on the US soil. So, this individual somehow managed to get an

invite to something. I don't know if it was an interview or some other event like you mentioned. And so, he took a flight to San Francisco. And guess who was waiting at SFO gate number whatever as he arrived? The FBI. So, he was immediately arrested and uh sent to jail and had some fines uh and had to recoup a number of the costs, the profits of what

he he gathered after this attack. So anyways, really cool story around collaboration, really working effectively between IR, red team, and legal in this case. So now I'm going to pivot to a story where the key theme is that sometimes collaboration isn't planned. It's actually forced on you. So at one of my former employers, I was running a red team and [snorts] I was compromising systems as a

red teamer should be doing. And then I came across a system was that was a little bit different than the rest, right? I saw it was vulnerable and I exploited it and I realized well I didn't realize I found out later that I compromised an attacker honeypot that was already on the network. I didn't know any better. So I kept about my day-to-day activities and what was

really interesting here was that this is a at this point right and they knew I was red team. So they were clever enough to leverage my identity to then launch attacks as me to blend their traffic through my box and my account. And it's really hard for a defense team to then differentiate which one is Farson or which one's red team, which one's AP. By the way,

apologies. I know the mic's dangling on my ear a little bit. So if it comes in and out, I'm trying to be mindful of that. So how did I come to know about it? Well, one day I get a knock on the door at this building and a somebody dressed really formally like an investigator comes in and says, "Are you Farszison?" And I said, "Yes." And she

closes the door and says, "I got a couple questions for you." And she asked me at 3:00 a.m. last night, "Were you accessing the machines of these three executives?" And I was kind of floored because I didn't know what the heck she was talking about. and she knew. It's not like she's coming in guessing if I was the attacker or not. They had enough evidence at this

point. So, but what she did at that point really kind of resonated with me and left an imprint on my career because she could have talked to me as if I was the victim or if I was an idiot redteamer that just was, you know, didn't care about hygiene and left my credentials everywhere on the network. But she took a different approach and her approach was to

just tell me, "Hey, Farzison, there is evidence that an AP's on our network and they targeted your account, right? But we would love your help to monitor them." So that was the approach. So instead of making me feel like a like stupid, she brought me as part of the broader IR initiative, which was really magic in a way. and she probably doesn't even remember who I am,

but she left such an imprint on my, you know, how I approach conversations moving forward. So, she asked me, uh, just keep going about your business to your day-to-day normal routine. Don't do anything too sensitive. Just log into your email, etc., and pretend everything's normal. So just imagine that for a moment that you know every day you're going to work, you grab your coffee and you're ready

to to get to some email and you know on one side of the machine you have a forensics investigation team watching your every move and on the other side of the machine there's a also watching your every move. There's nothing normal about but I had to keep that up that act up for about two weeks. And during that time off out of band the forensics team told

me hey Farzison login over here. Hey Farzon do this and then wherever I went the attacker went which was great. So we started generating and collecting all these cool IoC's through that exercise. So it was a net positive because of that collaboration rather than just an incident. And that's sort of the catchall here is incidents are opportunities. You hear this a million times right? So really leverage

them. All right, we got two more stories here. This one is an interesting one because this is where collaboration broke down, right? And turned into one of the most humbling moments of my career. So um I was on a red team at Microsoft at the time and I was running a pentest of a number of HR related applications and part of this was um a particular application

that disclosed sensitive data related to uh you know employees specifically their salaries. So I found a vulnerability that allowed me to get access to salary information. Great. I stopped and I reported the bug. But I made a mistake. I instead of stopping at one record, I decided to write a script that enumerated a thousand records like an idiot. Um, in retrospect at the time, I thought I

was doing something good like I was showing scale of impact, right? Which is true. You need to sometimes demonstrate that during a red team operation. So, that was one mistake. The other mistake is I was feeling really full of myself at the time, you know, finding this critical issue and I decided to kind of brag about it and I made a joke to my office mate and

I said, "If there's anything this pentest taught me, it's that I'm the lowest paid security engineer at Microsoft, right? Just just to be chummy." And he laughed, but somebody down the hallway heard that joke and he didn't laugh and they decided to escalate it up the chain. And about an hour later, I got a call. Well, it wasn't a call. It was a message from my manager.

And he said, "Farzone, I need to see you in my office." And I'm walking into his office and I'm feeling really good. And I'm thinking like this, my manager is going to sing my praises. I'm up for promo this year. This is going to like lock that in. And he just like a very stern face. He just said, "Farison, I hear you're looking at people's salaries. What's

going on?" And I was it just I kind of like couldn't compute the conversation in that moment because I was just on this high of finding a critical issue and how could that not be acknowledged right now, right? And he's like, "You're looking at people's salaries and based on what I'm hearing, we have to or I already referred this case to legal as a case." and an

ethics case was slapped against me during my time at the company. And he said, "Best case, you lose your promotion. That's best case. Worst case, you lose your job because of this." And I it was such a weird feeling because it was all within this 10-minute window. I went from feeling like I had this amazing critical find uh and one of the best uh uh like vulnerabilities

I discovered in my career to all of a sudden feeling like my career is at risk So this is kind of I think transcends beyond just red teaming, right? If you're in a role with administrative privileges, um, and you have the ability to, you know, have visibility across employees or the network, the real advice I I give here is just like just because you should doesn't just

because you can rather doesn't mean you should, right? And if you have to think about it, you probably shouldn't. So really just double think before you you take an action. If you're just curious, really think if that's necessary for your role. um you know scope is that legal permission but trust is the social permission and you really need to have both in order to be successful in

your role. So that's that's that negative time in my life um promotion to reel etc. I managed to keep my job though so that was a good that wasn't that positive. Uh this is the last story I'm going to share here and it's one of my favorites. It's actually puts me in a good light for a change. But it was um this actually very kind of local

here to the Southern California area. There's a very prominent entertainment conference. I can't name the name uh just because of an agreement with their legal team, but they have around 70,000 attendees every year when this was hosted. Um and it again big on the entertainment industry in this area. So you could probably guess what it is. So what happened was they were dealing with incidents where tickets

were going missing. Like people who purchased legitimate tickets at this conference all of a sudden lost access to them and they couldn't log into the portal anymore uh to even see what happened. And at the same time you saw a spike in ticket sales on reseller sites like eBay. So like maybe the two are related if you if you do some analysis there. Now I myself am

not involved in this investigation at all. I don't work for this conference or this organization, but I happen to be going to this conference that year. And it's kind of this this uh great story of surprise collaboration where uh it's just there the opportunity is there and you just step into it and you have to just find a way to collaborate with people you've never worked with

before. um there's around 70,000 people. The tickets cost around 300 to 900 a ticket. So that's around $35 million of impact if all these tickets were able to be hijacked. And that's exactly the vulnerability I'm going to be talking to you about here in the next slide, how that exactly happened. If the conference were canceled, all the sponsors were to step out, it would be 2x, 3x

this cost and impact at least. So again, rather than tell you what I did, I I'm going to try to make this a little bit more fun and interactive in this last story and see if you can spot the bug. um because ultimately this vulnerability was allowed me to demonstrate the hijacked of everybody's ticket at this conference. So you're not going to read any of this and

that's intentional. This is just a confirmation email once you go to this conference that you receive if you purchase a legitimate ticket. On the right there's a link you click in the email body that takes you to this web page. And this web page tells you information about your ticket. Remember, you just click the link and you're auto logged in. And you see my information, my title,

etc. My tickets here and that's it. Now, if I were to ask you that there's a vulnerability in this platform, it's not necessarily too sophisticated, but it allows you to hijack somebody else's conference ticket. What would you try doing? Does anybody have an idea? Yep. IDOR. 100% IDOR. So, it's again not super sophisticated here, but IDOR will work. Um, let me highlight what's interesting. So, I know

it's hard to see in this room, but what you see in the URL here is two query string or three query string parameters, but the two of interest is login equals FK followed by a bunch of numbers and then password PWD is equal to some number. Does anybody have an idea what to change that to? Any takers? going once. >> Yes. The FK are my initials. Very

good. >> Yep. Uh what do you think the number is? Yes. The PWD is maps to the registration number in the web page. Right. So that's an innumerable sequential ID. Everybody goes up by one. You got my initials. What's the rest of the number in the the login part? Well, I hope nobody knows what that number is, but I do. And I'm going to dox myself for

a second. That's my phone So, all of these are guessable parameters, right? And what made this worse was that the phone number was actually a throwaway field in the login. The primary key was just the initials. So, all you really need is to build a script for the initials that go from aa to zz and an enumer and a a script that goes from 1 to 900,000

and just enumerate every possible ticket you could hijack. And I did that during a five minute uh proof of concept and I really wish I captured a video of it uh but compromised hundreds and hundreds of tickets within that fiveminute window. Right? So, that was really it. nothing too fancy and but the good thing is it got me in the room with this organization I never talked

to and spoke with before and it led to real incident containment and it was just awesome to see this organization step up the way they did and immediately implement the fix and uh handle the whole fraudulent cases that happened on eBay and the reseller sites. Um, how much do you think I got paid for this bounty? A little bit more. I got a t-shirt. I got a

t-shirt. I had to work hard for that t-shirt. I remember telling them my size and uh they sent me extra extra extra large and I kindly followed up and said, "Could I just get like a medium?" Like somewhere in my range. Um but yeah, at least I got the t-shirt. >> Very good. Very good. Sorry, I should have played the final animation. just changing the initial to

RO from FK, keeping the phone number the same, and then changing the reg registration ID by a couple digits, and bless you, and you get the um somebody else's ticket here in the bottom, some guy named Russ. So, again, scales to impact everybody at the All right, so that's really the lesson from today uh and the thread running through all the stories I shared with you, right?

It doesn't matter whether you're a defender or on the offensive side, a software engineer or a CIS admin. When we stop treating each other as opponents, we win. And guess what? We start trusting each other more as a result. So again, I I mentioned this at the top, but the real vulnerability at companies isn't a technical flaw. It's the human dynamic. It's human isolation. And the real

fix or patch is just being able to work together united against the exploit. So, thank you all for having me. Um, enjoy the rest of your day at Sunset Con and Scale. [applause] Marco. Um, if you I can step away if there's questions I could take him to, but I know we're we're wrapping up. Uh, any questions I guess for the moment? Yeah. Um, the most important

thing is just keeping people in a conversation, right? So, right now I have a detections team as well as a red team in my organization. At other places, they would be siloed. Now, I have them all be part of the same call. And yes, there are needs for them to have their own individual sessions, but once a week all the teams are collaborating and just when you

have the open communication, the walls kind of come down along with that. And it's just really a culture thing. Then people are more comfortable saying, hey, like I I'll give you one example uh of a recent red team where we found an opportunity to like pivot from a guest wireless network to do something bad, right? rather than just save that and then drop that as a finding

at the end of a report, we notified the blue team right away and they wrote an immediate detection on uh port scanning and guest wireless etc. And at the end of the day, both the red team and the blue team got the credit, right? It wasn't just one team's credit, it was both teams credit. curious. >> Just curious about your um your last story, you know, in

terms of responsible parties there. Obviously, the conference didn't roll their own e-commerce system to deal with the tickets. >> Great point. I mean, now, you know, who did they license that from or did they actually just vibe code it? Right. You just you you noticed something that was really important that I missed presenting in that slide is that because it was word out there was that was

a third party site that handled conference ticketing and it didn't just impact that conference it impacted all the conference tied to the platform. So the the financial impact was much larger and it goes to to show that you know you could have a security team doing everything right and if you contract somebody that's doesn't have robust security you run into flaws like this. So that company actually

wounded up going out of business shortly after and I wonder why. Yeah, good question. Anything else? All right, thank you all. Appreciate it. Thank you [applause] just in case. Uh, so it starts at 11. So if you've been here continuously and want to go for a bathroom break or grab something outside, you have those 10 minutes. Um, and now hit the doors. Looks like it has a

good Someone set up a bond. We get [music] signal. All right, folks. Uh 11:02. We'll give one more minute, let folks drag in. Um so for our 11:00 talk, as all keyboards belong to us, uh Federico Luchiferi. Um he's a product manager product management director for self storage at IBM and Red Hat Co-ato Orali Picari book on AWS systems administration. So uh without further ado, please give

a round of welcome applause. One, two, three. >> Is this too loud? I'll try not to get excited. So, [clears throat] as I was saying, this is my Defvcon presentation from last year. So, it should be interesting. Many of you know me. I've presented at scale for 10 years in a row now. So I'll keep the introductions brief, but for those of you that don't, I've had

the privilege of spending my entire career in free and open source software. And as the track chair was saying, I'm the product management director for Seph Red Hat, uh, IBM, and a bunch of other places. Uh, previously, I was the Ubuntu server PM at Canonico. And if you go back the decade, I was infamously known as the system management tar at SUSA. Um the talk has nothing

to do with my day job. I'm a manager these days. Um but as a former embedded developer, I have an interesting idea of what should be fun. So you could say that I do this to prevent my brain from rotting completely. Uh I'm also the co-founder of a small R&D shop in Boston focused on sound and computers. U so for us this is doubly The humorous obligatory

disclaimer that goes with all my talks is that usually we risk breaking some hardware and it would come out of your pocket or some such. Um no liability if you stub your toe or break the end of the univer or bring about the end of the universe kind of stuff. But this is a completely different no liability disclaimer. It is uh this talk is uh adjacent to

things that are classified. So we want to be very clear that this was entirely compiled with open sources. I'm not disclosing anything that's secret here. As a matter of fact, I'm not read into these secrets. So I cannot accidentally tell you because I don't know. Uh which is good. Um, and then the usual disclaimer for hacking talks, which is obviously you know what the law is. We're

all grown-ups, so don't do anything bad, but also um uh be nice. Come on. The world is already messing up as it is. So, we have a lot of stuff in this talk. Let's get going. Uh there seems to be a wild amount of stuff going on in academia. Maybe it's not uh being used much um uh out there. Obviously, we have no idea what countries are

doing to each other, but in terms of industrial espionage or what's going on in the industry, we have a rough idea and it doesn't seem to be a major concern at the moment. But in the you know when that when the uh academics are going wild, the the nation states are at least a couple of order of magnitudes better at at the game. So uh there must

be definitely something serious going on um between countries and we know this has been going on for decades between countries. Now now we are not focusing on signal intelligence across the spectrum. We are just looking at keyboards. Um keyboards have direct access to the plain text of what we're so carefully encrypting. They have access to our passwords before hashing. um they have access to everything very easily.

So it follows we obviously want some exploit action and there is some pretty wild stuff out there. Uh let's go and see what is going on. So first let's start with something light. Eric Hasselton interesting career trained in neurology and became a Disney engineering executive then and eventually he was recruited by General Hayden to run the NSA's R&D department. A pretty original career trajectory if I ever

saw one. So the first connection to movies is Disney obviously, but um there is a second one Um, this is from Hasseline's book. A great book by the way if you enjoy technology and espionage stories. And these are all real. Uh, it's very enjoyable, well written. Details among other things the discovery of IBM select electric implants um implants in IBM select electric typewriters in the 1970s which

was a major coup for um, uh, Soviet espionage. And at this point he is talking about the origin story of Charles Gandhi who actually discovered those implants. So legendary uh researcher Charles Gandhi how did he get his start in the business? He went to the movies. FBI FBI agents monitoring Nazi spies in America remotely beamed energy from addition antenna at the room where the spies were planning

their next attack. The intrepid FBI agents were able to decode voice signals in the room from reflections of a vibrating speaker in a telephone headset and worked the attack. And so from there, Gandhi decides to um go into the business of protecting the United States. But it's 1940s. It will be a while before the NSA is there. And eventually he becomes uh an NSA I've been really

trying hard to figure out what this movie was and I haven't been looking for a couple of years. It has even foiled omniscient chat GPT who told me that the best reference is a talk given at Defcon by me and when I explained that to chat GPT it was the closest I've seen a large language model to amusement. Um but so the hunt is still open and

there are a lot of FBI kind of propagandish movies from the 40s so it's not so easy to find. So young Gandhi was very impressed and went on to join the NSA. Now the Tempest code word bring us the first attack vector which we can locally refer to as van freaking which is somewhat of a misnomer uh but it's a pretty cool name. Dutch researcher Vinv opened

Pandora's box on yam signal emissions with his 1985 paper publishing what sigint professionals had known since the 1940s but kept classified. It was in the open that signal leakage existed but it was generally assumed to require expensive equipment and expertise. In Vanex attack he exploits the emanations of un unshielded CRT screens. Um we presume that most of the Intel community was targeting uh teletypes but same high

level of energies. Um so CRT screens UHF band which for those of you uh less than 25 it's old overthe-air television. Um the monitor synchronization pulses aren't there. So he spent a fortune $15 to reintroduce those synchronization pulses and off he goes from a 100 meters away up to one kilometer he is able to get the TV what's on the TV of somebody uh people freak out

about this about every 20 years I find as generations uh and so the next generation does the same unclassified NSA's histories say say that this rediscovery happened in the intel community as well um Bell Labs originally discovered this in the 1940s and then the CIA rediscovered in the 1950s and um ban freaking ahead Another big moment in around 2000 when um uh Neil Stevenson's Cryptonomicon gave it

a premier spot in in the story line, at least amongst geeks. So, it's been 20 years. It's time for you to freak out again. Now, uh here's the BBC video. Um probably we don't have time for the whole thing, but it's on YouTube. It's a decent introduction to what's going on. um they drive an antenna van down the street and as one does they read a letter

of some unknown professional eight floors up in that building. This was the 1980s. It was probably not illegal or they conceded to read um whatever somebody that was in cahoots with them was doing >> the access code to Prince Phillip's private computer mailbox and left a rather unwelcome birthday greeting. Well, that same hacker has now linked this terminal to the computer at Daresbury, which is used for

nuclear research. At the moment, you can read all this data because it's obviously low-level security. But if I try and go deeper into the computer, the data will be encoded. So, all I'll get back is this sort of information, what's known as shash. To be able to read that, I'd need to have access to a decoding system. But we found out about a particularly worrying method of

eavesdropping that gets around even the most advanced encryption procedures by reading the operator screen before it's encoded. The way of doing that is based on a phenomenon that's been known about for at least 20 years. And although it's not actually an official secret, there was no one in this country who was prepared to talk to us about it. The phenomenon is quite simply that all electrical equipment

gives off signals. If you take even a pocket calculator and you put it near to a radio which has long wave on it, you can hear the interference as I press the PS here. Well, even if a calculator radiates a signal like that, you might still be surprised to learn that a computer terminal radiates a signal that can be picked up over 2 miles away. What's more,

that signal carries the information displayed on the terminal screen, and that information can be read The data on this VDU screen is being transmitted all around the studio and it's being picked up by this receiving system here that's not linked to it in any way at all. It's only 100 pounds worth of equipment. So, uh, it's not the greatest picture and I'll try and stabilize it a

bit there. And if I move the aerial, you can see that the information disappears off the screen. Move it back and it returns. Now, all we've got here is an ordinary television set with an aerial and an amplifier and a rather special, if cheap, box of tricks. But we had to go to the Dr. Nail Laboratories in Holland to get hold of it. Incidentally, they use this

equipment quite legally to investigate this very problem. And they were the only people, excuse [clears throat] me, who we knew who could find out and would admit to having it. Well, we've seen it working in the control conditions of the studio, but yesterday we set this up to see how it would work outside. Well, at the moment we're outside an office block in West London, and we

found that there's a VDU screen operating about halfway up the building, probably on the seventh or eighth floor. And in fact, it's a letter being typed out on that screen. I can actually read it here. It says, "Dear Mr. Jackson, >> thank you for your Whoops. Thank you for your letter requesting the use Whoop! I've lost it again. Requesting the use of our facilities. I have spoken

to Mr. Saunders and he >> So that gives you the idea that Vanak himself actually built that equipment for the BBC because presumably everybody in the UK had signed the official secrets act when and when they asked they wouldn't talk about it. Unfortunately, they didn't find James. Now um here tempest code word refers to a classified protection standard for unintended signal emissions leak through EM or other

channels. Most common sources are or wear CRT monitors. Here are an exemplary from my retrocomputing research for those of you too young to know what the CRT is. That's a monitor for a Macodor 64. that much we know because the code word definition itself has been declassified in 2008. Much of it is blanked out but a few interesting things were discussed. Discovered in 1943 at Bell Labs

reading and encrypting machine from across the street rediscovered in 1951 by the CIA. Strategies for mitigation include shielding, filtering, masking. Eventual resol uh resolution being the instruction for facilities to control an area of 200 feet in all directions. So if you had a station handling classified material, you were supposed to guard this area. Not notably in the NSA history, they say that they set this as a

practical target because they thought that a base could actually do that. that if they had requested for more, their personnel requirements would have been excessive. And the NSA hasn't declassified what the signal or what um uh corresponding range at those um voltages would have been. So we don't know exactly how safe this is, but this is the directive that went out back then. And they also requested

that um sites operate 10 teletypes at once in the same room. Um uh this is what we would call masking which today is generally understood not to work but it was probably great for teletype vendors I guess. So the strategies for mitigation here include shielding, filtering and masking. So shielding, put the machine that's creating the signal in a Faraday cage effectively. Filtering, trying to shield the frequencies

that carry the signal itself. Masking making so much noise that nobody can tell what the signal is. But with digital analysis, masking, you can consider it gone. Another uh bit of the classified memo material um is that it discloses what's called red black separation requirement uh which is illustrated here by another good book Peter Wright's MI5 memoir. In the 1960s the UK government wanted to enter the

EU and the French president General De Gaul had a monumental chip on the shoulder about the British. So as one does, the foreign office asked uh MI5 to wiretap the French embassy to understand what their negotiating position would be in advance. And as they uh they do so and as they examine the cipher text coming down the line, they see a faint signal much weaker than the

cipher text peeking through. And it turns out that this is the plain text. The French cipher was good and as far as we know GCHQ couldn't break it back then. Uh but they didn't have to because the proximity of the machines handling the plain text and the cipher was such that noise from one was crossing over to the other and going out of on the outside line.

So hence red black rules keep clear text away from equipment that may unintentionally or unintentionally pick up uh the signal. Uh the NSA hasn't disclosed what the red black rules are. It's just that they exist. Well, they did disclose something about wiring that they want wiring done a certain way, but probably not applicable these days anyway. Um by the way, obviously did no good to the UK

case. uh General de Gaul stayed on his position and the UK could not enter the EU until the Gaul passed away in the 1970s. But if you know recent history and this is funny um in a strange way um the world is strange I guess. Additionally, [clears throat] the declassified code word entry described signal eliminations in other domains other than EM seismic pre presumably referring to vibration

but only the title of the section was released and acoustic acoustic quaintly termed uh phenomenon number five for reasons that we don't know. So here is another target uh electric typewriters. This one is from my IBM Obviously here we have the EM emanation problem. But the interesting bit is the sound emanation. These things make a serious racket. It turns out you can figure out what is being

typed by listening to a typewriter or teletype sound. And the Tempest memo itself indicates this can be done from a 100 ft away with a shotgun parabolic microphone. Another interesting nugget, contrary to expectations, soundproofing a room helps the attacker, which if you think about it, makes perfect sense. It's a ratio between signal and noise that matters. If you soundproof, you're eliminating the noise. So, you're making the

attacker's job easier. Um this is mentioned I believe in the NSA memo but it's also uh been in the open literature for quite some time because pretty much every researcher in the area comes to this conclusion immediately as they start looking. Uh the range limits for all of these are dictated by signal noise ratios. So this is entirely predictable if you think about it. There is a

lot out there about the Tempest keyword if you Google on the internet and some of it may even even be true but um there is absolutely no point debating it because we cannot discuss it and we cannot even tell what is and what isn't. So some of you in this room uh who know cannot tell us and so there is no point for us judging the gossip.

Google it yourselves. Um, the other word you want to search for is pemin, which unlike Tempest has an officially disclosed acronym. electromagnates to something like unwanted electromagnetic radiation and interference. Just remember there is more than EM audio vibration. Uh there is more than EM audio vibration optical are all coming to the party. So let's start with optical. I will take revenge of the caps locks for 500

reges. I must admit I was disappointed to find no reports of signal leakage from the keyboard LEDs. Uh that is unintentional leakage. There are multiple Xfiltration research papers showing reasonable bandwidth in defeating air gap systems by flashing caps. I recall one reaching 56 kilobs. I don't know if somebody managed to do better. A character in Neil Stevenson's cryptonomicon exfiltrates data using in a novel that as I

said makes extensive use of van freaking in its plot to begin with. And then there is this uh we have an entire industry of pent testers and the gold standard for them is a keystroke injection tool disguised as a USB drive which is this the rubber ducky. If you have been to my talks in previous years at scale you've seen me use this. You should know about

it unless you've been living under a rock in some of the outer planets. There are USB drives that can disguise themselves as keyboards. And so the USB rubber ducky is the original keystroke injection attack tool. It's 15 year old old 15 years old at this point. Also the hardware has been updated. Um and while it looks like a USB drive to us, it li it acts like

a keyboard when talking to the OS and typing over 1,000 words per minute. Windows are and go. Specially crafted payloads written in a custom scripting language mimic a trusted human user while entering keystrokes at superhuman speed. It is named this way because it quacks like a keyboard. It must be a keyboard. You have the shell. You can just type your way to success. Uh you don't need

a crafted overflow. You just type on the shell and go from the keyboard. You can always start um a shell on Windows as I just said Windows given that Windows users have a punchant for running as administrators. Um but what enables the attack hampers data Xfill because surprise surprise you cannot copy data to a keyboard. This thing has storage obviously but it's a keyboard for the OS.

So, if you ever look up a rubber ducky exploit, what they do is typically push data to some dark corner of the internet, probably over IRC. Um but this was this changed in in 2022 because to reduce the price of PC keyboards IBM delegated the management of the cap state and lights to the host giving us a side channel uh for or rather for the ducky uh

that can now use to exfiltrate data uh to its internal storage. Side channel for decades in the making. key stroke in reflection relies only on caps lock num lock and scroll lock to establish a data transfer path. This is something a rubber ducky can now do out of the box. It's not something that you have to craft yourselves. So no more need to connect to IRC servers

on the far side of the moon uh to exfill data unless for some reason you want to. But wait, there is more. for one simple price of 1999. Oh, never mind. What is this? The lower line is an RS232 from the port. The bottom one is the signal led by read by uh Lafrey and Umpress looking at the brightness variations of a modem's TX LED. So we

have made our way to 2002. Now this was known as their optical tempest paper which was a pretty spectacular one at the time. Um modems are obviously a thing of the past but you could read the transmission from across a room and likely across a street with some tweaking. One bright spot in their study is that their testing examined hard disk LEDs and found no IO signal

leak there. But wait, there is even more. Uh the previous team was from Loit Martin and they were topped in the LED category by a team at Benorian University of the NEGV uh with the new glowworm attack uh which exploits flickering of LEDs in the room uh in the target room connected to a sound speaker. So, in this case, it's either the speaker itself, if it has

an LED, or if it's a USB powered speaker, the LED is on the USB hub powering it. And they were able to extract audio from up to 25 meters away. Um, I don't know if the picture from their paper is clear enough, but you can see that there is what looks like a 10-in telescope in their lab right there on the right, and they are shooting down

the corridor. Um, this is not itself a keyboard attack, but it leads us to one. Uh, the joke used to be that you don't drink and drive, the math joke. But now the new joke is don't Skype and type. A team led by a researcher at the University of Rome, Lasapiensa, demonstrated what hypothesized and Asenov demonstrated as a possibility. The audio emanation of a keyboard can be

exploited over a phone call or as it turns out a voice over IP one these I must say I like how their code is laid out. By the way, this is not the usual university R&D complete mess. It actually looks like code. Uh it shows that at least somebody on their team is a CS uh person for once. If you type in Skype, what you typed can

be recovered with top five guessing 91% of the time um according to their paper, according to their method. Uh but to do this uh you need to know something to tune the model, something about um the typing style of the target and the keyboard, which leads us to um so actually this is interesting in a different way. This was um 2018 and as I as I said

has had predicted that this would be possible about 20 years earlier and Asenov had demonstrated that the right signals went across less than five years or maybe couple of years after said it then 20 years later it's actually been done in academia. So nice verification but obviously that also means that uh the Intel community has had this forever. so who is this Mr. Uh at the turn

of the millennium professor Marcus at Cambridge published a massive report on CRT emissions. It's so huge that at at some point I thought it was his PhD thesis. it is so large um uh that yeah it stands out you think it's a thesis and this is likely to remain the definitive work for EM emissions at least very likely until the NSA declassifies some of their internal materials

um I guess CRTs are not so common anymore so the signal footprint for for EM is not as strong as it used to be but the sensing side is also more sensitive. So, who knows how that race is going. But regardless of when the next big paper comes out, this one enshrined as Vanek's successor as the spiritual guide of non-classified uh signal leak researchers. and uh basically

the academic community used him as a clearing house of information of what new findings were for decades. And later um a couple of years later uh the other name that I mentioned two IBM researchers Asunov and Agarwal published um while Coon's stuff is mostly um CRT and teletypes um Asunov study is um is So Asenovan and Garville systematically studied the problem and wrote a meticulously detailed paper

answering many core questions. In their attack, a microphone is placed near a keyboard from half a meter to 15 meters away and sound is sampled with a standard sound card of the period at 44.1 kHz. So probably a sound blaster pro. I guess the paper doesn't say. A neural network is trained to classify a labeled training set of 100 samples per key and is tested on a

second set. In a two key test, which is their first experiment, distinguishing between the keys K and uh L on a querty keyboard, the system performed flawlessly, the model scoring 20 out of 20 and averaging 95% success rate in longer tests. Um in the short test there were no false positives which is also interesting. It's not only correct it's also not uh not giving you false positives

for for a given entry. Using their setup uh they proceeded to answer core questions which make which is what makes this paper a classic. It's very methodically dissecting the problem. Will distance matter? They tried up to 15 meters away with no decrease in recognition quality. They placed the microphone behind the typist to see if that kind of obstacle would change anything. In another test with 30 keys

and 300 uh entries, uh the model guessed the correct key 79% of the time, rising to 88% in if you use top three choices, which is kind of become the standard for these studies. They then tried the model trained on one keyboard on another unit of the same uh and the recovery text was poor but the model still yielded significant entropy reduction. So you could have used

that attack to compromise the entropy of a password although maybe not necessarily to retrieve plain text straight up. In another test, the typist applied variable force in striking the keys. Um, the fixed force model floundered, but then they trained a new model of on variable force, which was successful with variable and fixed force alike. In yet another test, they subjected the variable force model to inputs from

multiple typists uh who were um free to type in any style they chose. Uh the quality of classification was adversely affected but only very slightly showing the applicability of a model trained by the attacker to a potential victim. Uh studying the average sound of individual keys, they narrow down the fact that the individual locations on the keyboard of a given keyboard unit, not its model, not the

typist style, not the keys themselves. They plucked the key caps and moved them across the keyboard to see if that made a difference. Or the switches, they did the same thing. No difference. Um, what distinguished the sound is the place where you strike the keyboard. So, you can think of it as a symbol on a drum set and where you tap makes a difference to the sound

and the the keyboard is your plate and where you're tapping makes a difference. It's like drumming in a in a very real Um then in 2005 a UC Berkeley team led by Liwang puts the final piece in place. They revisit the IBM study and demonstrated the use of unsupervised training by leveraging inherent statistical constraints in the data. The model is uh designed to attack passwords and can

break five character random passwords in fewer than 20 attempts and 80% of 10 character passwords are found in fewer than 75. You would think the game is already over but we are far from it. It gets way worse. In 2023, a team of UK researchers demonstrated 95% accuracy without use of language model constraints. these models that recover plain text make assumptions usually about the language that you're

typing in, which then lead to assumptions about the statistical likelihood of uh combinations of two or three letters in that language. This model did away with all of that. uh I don't remember where in the UK this team is based but they basically said forget it that it can be any arbitrary text and what they did is they recorded from a nearby smartphone that was just placed

on the same table as the keyboard and worst of all on a difficult keyboard too. uh the shallow shallow Apple MacBook Pro keyboard that people like to complain about which is not one that makes much sound. So uh they had 95% accuracy with the setup uh dropping to 93% when performing this feat uh over Zoom. Um make your piece says cats about this one. But actually you

can get one bit of good news which is that Zoom has introduced filtering not for security reasons but for their own reasons. Noise background in the call I think in 2018. So this attack would no longer work. The the filtering that Zoom applies to to for noise reduction would would break But if I haven't completely crushed your soul yet, I have one more. Uh, in the spy

phone attack, a Georgia attack team successfully attacked the audio emanations of a standard keyboard and retrieved 80% of the source text using accelerometer data. No microphone uh source from this setup here, similar to the other one. Now the difference here is that the microphone is privileged in in any smartphone. So you have to hack the smartphone to get to the microphone. So first you have to have

the user install something and then you have to exploit the phone. The accelerometer is not. The accelerometer is unprivileged. So all that you have to get is people to install a stupid horoscope app and off you go. You have access to the accelerometer and you can read it this So, Texas filtration can take place without compromising the phone security at all. You just have to install the

Trojan horse app. But let's not forget our friend Vanek. Uh, this one actually surprised even me. I put him in my abstract because he is a cool fellow and I needed him to explain the concept of eminations to uh an audience without sigant background. Uh, but I didn't really expect that we could do this against keyboards and boy was I wrong. Um, I should have seen this

coming because keyboards are really serial devices. So PS2 serial 5 volts not that Um, in 2008, a team at EPFL in Loausan retrieved 95% of the keystrokes on PS2 and USB keyboards up to 250 me uh up to 20 m distant even through walls. uh they used the M signal leakage and the loop antenna with early versions of GNU radio uh because they couldn't afford the classified

equipment so they did it on the cheap much what's much what's more they demonstrated the ability to distinguish different signals in the same room uh basically demonstrating that masking doesn't work and so the 10 teletypes trick is out the window I'm afraid uh the figure above sh uh shows data clock and the emanation at 5 mters. Uh this is also an interesting paper to look at if

you want a tutorial on GNU radio that is not designed for double E uh brains. Um so if you don't want to get a double degree in electrical engineering before learning GNU radio, this could be a good way in. Now, I've convinced you the world is full of these uber smart adversaries and there is really no hope. That won't do. Let's look at good old-fashioned screw No.

Uh, well, there is a wireless keyboard. I think it was 20. It was around 2000. There was a wireless keyboard without encryption at all. So, that's plain stupid and we are not going to pounce on that. But, um, can't fix stupid. But uh while I was thinking about the keyboard leakm example to show um I had this sitting right in front of me staring at my face

and then I was reading the Bastil security mouse jack exploit and as I read I realized that I thought that only micro only uh Logitech mice were affected but in reality the vulnerability is in the chip it's not in the vendor. and I thought that vendors had sorted that mess a long time ago. Uh, but then there was this on my desk and so yes, pawned. I

believed that uh the advisory affected only Logitech, but I was wrong. The dongles affected by the mouse jacking attack encrypt keyboard communications as they should, but not mouse communications. which is less great. I guess the logic was we want the mouse to be very responsive. Don't introduce delays. If somebody wants to hijack the mouse, who cares? Maybe not completely unreasonable, but there is a problem that whoever

made the requirements did not communicate clearly to whoever wrote the code. The attack has a keyboard connected to the dongle as a mouse. Well, it has a computer connect to the keyboard pretending to be a mouse. And once you have the connection in plain text, you can call the keyboard API and inject text. So yeah. so people still screw up on both sides. No need to go

down the rabbit hole if the barn door is left open. Now [clears throat] I had this funny image. Every time somebody presents at Defcon, they have weird stories about uh what happens when the hotel staff gets in the room and they get all weirded out by whatever strangess you have in there. And I had this picture in my mind of the cleaning staff getting in my room

and looking at how boring this hacker is with this desk full of And then even worse, wondering what kind of sociopath has 10 clacky keyboards, he must his colleagues must really love him. And now uh Gennady Garganov has posted uh to YouTube a tidy example of how a sound eminations attack would work. It is a small bit of web assembly. Uh so it loads in the browser.

You don't need to do anything to set up and it tries to predict what you typed from sound tries to recover what you typed from sound and statistical inference about English letter pairs alone. Gennady's model is not as sophisticated as any of the ones we discussed. Um it's just simple statistical matching. It doesn't have a neural net. It doesn't do anything u strange. and the neural net

would help quite a bit here. Not a huge one, just a couple hundred nodes. And this is really its charm. It's easier to tinker with and experiment before dipping uh in deeper into the pool of the subject. Um, this, um, hopefully the monitor is clear enough is an example of me poking at the letter. I believe it was I repeatedly. And if you read the tracks below,

it's first guess, second guess, third guess. And so you can see that there are really a lot of eyes in there. and this is a hobbyist's code, I want to stress, and it's basically already passing the 80% threshold without any particular effort. If we were demonstrating this live, this would be the setup that I would use. I'm not doing this. I know how loud this room is

to begin with, so I'm not going to try. But, uh, this is the way I usually test stacking the odds a little bit with a good microphone. Don't need to fight physics as well. Now, if we were demoing the constraints would be those the microphone would be very close. I cannot make any typos training the training set. Excuse me. backspace is not handled by that code and

I need to enter between 100 and 300 characters uh of valid English words to maintain the statistical uh validity and and off Now let's [clears throat] go back to the security aspect. It is a very realistic possibility for an adversary of medium sophistication to tap your keyboard's text in the clear before it is secured at rest or in flight with cryptography wrappers. Uh it's even easier to

decre your password entropy. I don't think that you have to worry about 100 meters range but I would worry around 40 50 at least. Now the tools to do this are not widely available. What is out there is primarily research material and not easily used for research purposes. And as I like to point out, until successful security becomes commonplace on the network level, why would I hack

you locally if I can hack you remotely? So for real attackers, this is probably not the most pressing concern unless you're you are in national security uh space. But in terms of industrial industrial espionage, I would still worry about the network more. Now a security analyst from Lincoln Labs brought me around to accepting that all eggs in one basket is really the only option for the defenders.

The usual logic goes, the attackers only need to find one hole in the perimeter to succeed. But the defenders need to cover all holes to succeed. And so to even the odds, defenders pull resources, which results in single implementations, which something that generally engineers hate. Why would you have one Bluetooth stack for the entire world or one SSL stack for the entire world? In in security, this

has become prevalent because the odds are so obvious. But in areas that are not security, implementations proliferate. And then this is why the the 300 thinking point is valid because you still have to convince people that one implementation that everybody is trying to make safe beats the variety because you cannot produce enough variety to make the attacker's job um difficult. Actually I think there is a DARPA

research topic on increasing the variety astronomically but that's a it's a completely different uh my thinking here is that it would be nice to have some kind of general purpose defense tool um that is some general purpose security tool like an anti virus that um uh that doesn't require you to have a clearance to deploy it or to purchase it or just make it part of your

standard um industrial safety perimeter. So, um, if you know of a three-letter agency SBI sponsor that could be interested in funding some research into better securing this attack vector, please let me know. I think that it's safe to say that all keyboards belong to us. Um, there is more. This is what could fit in 45 minutes. So um as I said it is really uh open season

if you want to attack this kind of This is my contact information. Um and as usual speakers are Pavlovian devices. So submit feedback, let us know what you liked, what you wanted more of, what you wanted less of and all of that. And uh with that, thank you so much. questions. Um, I think the chair has >> we have a few minutes if there are any questions.

as I'm told. >> Anyone who wants to stand and ask questions, feel free. >> Sorry, I'm not a computer. I'm an electric mechanic engineer for company. That was great. I like it. >> I'm glad. >> Wonder why they want keyboards at my place. So, >> any questions? Feel free to ask for the mic. >> Thank you for coming. >> Thank you. >> He answered everything. That's fine.

[laughter] >> I don't want to stand in front of >> you. You mentioned two books. One was Spy Catcher. What was the other one? >> Oh yes. Um so the question is I mentioned two books. Spy Catcher. Peter Wright is the one from MI5. Um uh Eric Hazelton. Uh I think it's the spy in Moscow station. Let me see. Let me I just go by the author

by authors to keep my sanity. But yeah, the >> [laughter] >> All right, >> last chance for any questions. Uh, okay, we have a question. >> If you don't mind putting the uh contact info, All right. Would you mind putting your contact info back up again, please? >> If you want the contact information, it's uh going to be put up on the board. Okay, folks. Uh it's

uh 12:02 and it's time to get our uh next talk started. So uh Bill Bington is a longtime activist, cryptography enthusiast, and a senior staff at EFF with public interest in technology team. So um it is my immense pleasure to introduce somebody from the EFF and a senior staff. So please everybody welcome a round of applause. Hello. Testing. Yes. Great. Awesome. Thanks everyone for coming out on

this uh first day of uh scale sunsack con. Um I'm here to talk with you about a little application that I wrote called AP Keep. Um and uh thanks so much for coming out. So, first of all, who are we? Uh, you may have heard of us, uh, but I'll give you a brief summary of what the EFF is and what we do. So, we're a 501c3

nonprofit based in the Bay Area a little bit north of here. And we comprise u this triforce of technologists like myself, activists who work against bad legislation getting passed and lawyers uh who fight in courts for our digital rights. And uh we fight against the expansion of surveillance technologies um that that we uh that we're constantly seeing. We're fighting it back against advise lawmakers, do outreach, um

public education programs, um and I said fight against bad legislations and uh and we also develop technologies for your privacy and You can see us at eff.org, ssd.eff.org, which has our guides for surveillance of defense. And if you want to donate to us, uh you can go to eff.orgdonate. So, who the hell am I? This was a a caricature drawing that is cut off at the end

at the bottom. Um, but uh this was done at uh Infosc Southwest. It reflected me uh you know, seven years ago, but I think I've filled out a little bit since then. I'm a senior staff technologist at EFF. I work on the public interest technology team there. Is there any way that we can actually get that resolution? Maybe I can just change my resolution or something here.

See if that works. No, no, better. Oh, yeah. Okay, there we go. I can't see it on my screen, though. Well, I'll just have to look look over my shoulder then. So, I work in the threat lab and cyber security policy working groups, which are kind of like our internal ways of uh talking with each other and figuring out what our policies are on on issues. Some

of my previous work includes uh HTTPS everywhere. I was the lead of that browser add-on from 2015 to 2018 um at a point which I'll kind of cover later uh involved uh some build out of its capacities and currently I work on panoptic which is a browser fingerprinting tool to help you uh figure out how unique and thus how trackable your browser is amongst all of the

brow browsers that we've seen uh in the recent past and I do some malware reversing and a focus on Android and this is kind of where my story starts with AP keep. a lot of people think that Android malware ecosystem looks like this. This comes from a company called Zerodium that went defunct last year and they were basically offering rewards for zero days that uh they received

uh and then they would give a cash payout for those that were disclosed to them. And these zero days um you know came in the form of like remote code execution with persistence um uh that was you know via a text message sent to you. Um and this is a valid you know concern especially with the private uh you know private um uh malware private spyware actors

um the paragonss and NSO groups of the world um really take a lot of value in the stockpile of zero days that they can offer to undemocratic regimes frankly um And uh this is one big way. But for the most part, Android malware doesn't actually look like a zero day that's sent to uh that is a zero click uh that's sent to your phone and compromise your

phone uh right off the bat like that. It comes from APKs as a initial infection vector most of the time. Um and these are and just Android packages APKs, right? And then this might propagate through the system. Um you know, an APK might uh infect the zygote process, which is what's cloned every time you open up a new app on Android. Um so malware might propagate that

way, but the initial infection comes from an APK that's downloaded and installed. So, here's a few blog posts that I've written in the past. Uh, one is in the family of what's called tour hydra, which is a uh piece of malware that uses the tour network initially to discover a tour hidden uh service and then gets a clearet URL through that tour hidden service uh in order

to find the command and control center. Uh why does it do that? Well, tour is very hard to shut down, right? It's very cir uh censorship circumvention techniques uh don't allow you to shut down tour nodes very easily. So this malware was using that fact um in order to make sure that well if their clearet site gets shut down then at least their tour hidden service would

still they'd still be able to like point it at another clear net site um in order to keep the botnet going. Um so that and then they uh used some interesting offiscation techniques in the malware dropper that I looked at. Um, this was masquerading as a European banking uh app. So, and on the right there was this piece of malware that was embedded in Android TV box

like set top boxes and it came right out of the box. And this uh is part of what came to be known as the Bad Box uh malware uh network. And it was installed on largely low-end Android uh settop TV boxes that you could get via AliExpress or uh or uh you know even Amazon and Amazon took a long time uh if they actually shut it down

at all. Um so we covered and confirmed some of the findings of an independent um and publicized it. Um and the uh there was a that botn botnet um in particular was shut down and then a inheritor of that botnet called badbox 2.0 know uh showed up and um that was the subject of a recent FBI uh notice on some of these types of low-end Android uh

boxes. this kind of malware research was the background to why I wanted to have some kind of a reliable way to get APKs in the first place. Right? [snorts] So just a you know kind of bird's eye view of what happens when you go and download an Android app I usually say on the Google Play Store. Um well, you store that file uh on your system with

a APK extension and that's an Android package and that package will run on either what's was called DLC VM in the past and now it is called Android runtime and a lot of times more recently you know it used to be in password one APK was equivalent to one Android package. Um this is no longer true. Um an APK can have like the base APK which is

the Android application and then multiple different variations which uh will uh contain local uh specifications or architecture different architectures that run uh via the Android's NDK which is a native development uh uh yeah environment. So, so you know largely Android devices um raw on these four architectures ARM 64 the8 traditional x86 architectures usually for and then uh army ABI 7. So these are the big four of

Android architectures. And why would you want to obtain an APK um in the first place, right? So it was useful for me for malware. Well, you have like a physical APK that's sitting on your disk, right? You could do analysis of that app and you could do static analysis. For instance, you take that APK and you run it through uh different applications which will reveal the Android

manifest which uh allows you to see the permissions that an app is is requesting. uh you can upload it to uh tools like Mob SF u which is a mobile analysis toolkit um that does some static analysis and creates a report of you know different libraries different permissions. It kind of creates this kind of comprehensive report. It's a easy tool to to use. Just upload an APK

and it uh it's a it's a it's a kind of way to to generate a good report for that. And then you can use command line tools like Jadex is pronounced Jaddx because it's Java DEX I guess. Um, and that will attempt to decompile that APK into what is roughly equivalent to the Java that it was written in. Um, caveat here is that it's often faulty and

then you can't recompile that uh into the application again. Or you can use a tool like APK tool uh which backsolies it. And basically what backsolying is is it it takes that application and then reverses it uh into its what's equivalent of of u of its uh well it's its Java bite code. Basically it's u what instructions are run on the Android runtime. And so that's all

in this thing called smally. And then you can change things on that. And then if you recompile it based on the back smally then that's a pretty reliable way actually. You can change instructions uh and get something that runs again. If you sign it then uh install it again. So you can modify code that way and then uh install it again. Um these reversing tools are pretty

useful if you have the APK. You can also do dynamic analysis on the app, right? Install the exact same copy on multiple places, for instance, see if it behaves differently, especially malware will behave differently based on the environment that it's placed in. Um, you can upload it to tools like Corellium, which is uh a good like virtual machine uh that that runs Android. Um, although it's been

bought by Celbrite recently. And you can compare different app variants. Um you know uh if you have an APK and you uh download a different say version of the APK uh how does the version on Google Play differ from other app stores? I've observed, you know, uh something in Huawei and the Android manifest contains Huawei specific permissions and it differs from the Google Play Store, uh Android

manifest for an application, the same version. Um so, so there's kind of interesting different variations that are published in different app stores and how does the app, you know, change its behavior uh when when you're upgrading a version of of that application. So for instance, you can see like on APK Fure which is an Android mirror or sorry Play Store mirror um there's all these different versions

of the Instagram application um that you can download and just do a diff basically um run APK tool on it reverse it into the backoly and then run like Google commit like or sorry um a git commit that and then just do the same thing on the latest version. Then see what changed there might be something interesting like you know oh a facial recognition uh SDK was

added to this app. Um so so yeah there's uh various kind of interesting things you could do with with you know checking different versions of that and seeing what the app development looks like but there were no good tools for it when I went and looked for something that was just a reliable way to download APKs. There was a few projects out there. They were kind of

in rough shape. Um, one of them I mean were was using uh the Google Play Store but it would fail half the time. Uh, others were meant to be used on device as an app but not as a command line tool. Um, so for instance the Aurora store is great but um it's basically a Play Store clone within Android like within the Android as an app environment.

So I wanted to me to have like a command line tool to do this and it was kind of interrupting our work in in threat lab. Um just trying to to analyze these different apps and just getting just having something that I can throw on a server and a way to download an APK from the Google Play Store or other places wasn't available uh reliably. So, so

well what do you do when when it doesn't exist? Create it, I set to create this thing I called AV keep and I wanted to have a few kind of I had a few design decisions in mind. Uh, first I wanted it to be written in Rust. My previous experience with Rust had been with HTTPS everywhere in what I called HTTPS everywhere live core. And what I

had done, well, just kind of as a background, HTTPS everywhere was launched as a browser add-on in 2010. and it was at a time when you know the login for Facebook was encrypted but the rest of the traffic wasn't. So it kind came as a response to this like a this um extension called Fireheep. Uh if you remember Fireheep, it was an extension which did uh cred

or uh cookie uh session hijacking. And so when you uh were at a coffee shop and you were sniffing traffic, you could use this fire sheep tool u to basically hijack someone's Facebook session. So in 2010 uh we came out with this HTTPS everywhere in order to make it so that if you're using a browser install extension, you're using the HTTPS version of that site wherever possible.

It wasn't a common practice at the time to just automatically forward you to HTTPS. Uh it certainly wasn't uh the situation we have now, which everything is um by default HTTPS uh unless you specify HTTP. Um so this was a time when it was uh much much less common. And so we wanted to make sure that people at least if they installed this extension that they would

go to the secure site uh if they if they desired and it was a pretty small subset of the web at the time in 2010 relatively. by 2018 um I had been maintaining this project for almost three years and the lookups to see whether a site had an encrypted endpoint had become snailike. It was so uh difficult computationally in JavaScript to do on this high level to

do to do this you know simple lookup. Um, and I figured there were data structures that were meant for this like bloom filters that you can check whether a site is actually just offers HTTPS as a symbol upgrade. Uh, an inclusion in some set of sites in a in this data structure. Um, and but there wasn't an easy way to do that within JavaScript. So I came

with a solution to uh move the core functionality of of HTBS everywhere that those lookups that lookup table into a new rust module um and include that in in the JavaScript the main you know uh add-on yeah use that bloom filter. How do you include some, you know, bit of say Rust code that's almost like systems level into an add-on? Well, you use something like web assembly.

And um there's great tooling in Rust for web assembly called was bind genen um that allows you to to write some rust and include it as a web assembly module in within um within a piece of like add-on or extension or website. And so the result of this was a dramatic speed up of of uh lookups. Um so you weren't delayed uh immediately if you're just trying

to access a site if you were upgraded. Um it was uh it was I didn't do any benchmarking on it, but it was way way quicker just noticeably. And it was great because you had this kind of systems level speeds and browser extension. So I had a good experience of rust uh based on that so decided to write this a keep tool in rust allowing us to

basically target all of the Android platforms like the ones I listed uh before uh and also um you know run that and this is a command line tool right so you're going to be running it not as an application but um on the Android command line the command line environment of choice which is termox. So and like you know the nice things about Rust is that it

gives memory safe guarantees uh via this ownership borrowing mechanism have type safety a strict compiler that kind of helps you along the way it's kind of strict but it helps you along the way of like arriving at uh code that you want to be writing and speed it's like systems level right it did require this cross compilation environment which was a pain in the ass But yeah,

it's doable. So, another design decision that I wanted to make sure was included was that multiple app stores, not just the So, we support Google Play, FDroid, and APK Pure. Now, why APK Pure? It's an interesting choice because it's a mirror of Google Play. Well, you know, the reason why I want to include multiple stores in general was because I I kind of uh saw APK as

providing, you know, versatile role. It's just a generic tool. It's not necessarily for malware. We're saying it's going to be used for backups. For instance, if someone wants to just download all the APKs that or the uh apps that they had previously on a phone and and then restore them or uh whatever the case may be, it's a flexible kind of tool for that. So, I didn't

want to pigeon hole the project in one for one particular purpose. Um you can't know why people want to source APK files. So uh support multiple stores you might uh want to download things from you know purpose or you want to you know mainline like Google play is obvious why you want to support that but uh there's this you know redundancy principle um that you can download

from a APK pure which is an APK uh mirror site um that mirrors uh Google plays stuff also So, you know, some purposeful um you know, for the purposes of obtaining open source uh software and uh and applications, you could use Froid uh and that's um you know um it's open source and it's uh completely but it's uh also allows uh some unit guarantees they do some

reproducible build stuff too within after which is afterroid mirrors, which aren't mainline uh afterroid. Um that will support the afterroid protocol. Um for instance, the Guardian Project has one of these. Um some geo regional uh app stores like Huawei App Gallery. um different app stores that you might want to support and what we targeted for this. So I wanted to also you to just be able to

include a list of apps with their versions in a CSV file or whatever and make it so that you can download those just easily. So and I also wanted it to be like easy to use bal system. So that's one of the reasons why in you know uh support APK pure is because it's not requiring credentials out of the box in order to obtain APKs you can

just download them right so support APK pure then you just run this command you get the APK for that that app via APK pure that's that's the default download source um even though these others are supported so default to APK pure no account needed so this is kind of what I wanted to have happen. Okay. And just some CSV and then start downloading them and great. Then

you're off to the races. Uh there were some implementation challenges. Google Play Store is closed source and there's little little to no documentation about Google Play protocol implementation. there were some reference APIs. Um there's a Python API that was u available to do this. Um but but yeah, it was somewhat outdated and um it used that first as a kind of a following the design patterns of

that in order to support Google Play. Um it in it involved a lot of dynamic network analysis to see why something wasn't working when it should be working. uh using man in the middle proxy and like providing a certificate that man the middle proxy uh generates in order to to analyze traffic and where there's some mismatch between my API calls and Google API calls then just kind

of try to make them match what Google Play's traffic looks like um as best I can in order to say okay well this is the same thing that I'm sending why isn't it working Right. Well, why wasn't it working? So, uh, Google was doing this really interesting thing, and I'm not sure why exactly, but they were actually using cipher suite fingerprinting in the API for Google's AP

Google Play API. So if you didn't offer the exact cipher suites that Google expected in the exact order that they expected, then it wouldn't work. Um, so it took a frustrating few days to figure that out. Um, and then I needed to offer the right ciphers in order to make it work. and that required OpenSSL. and so I needed to have OpenSSL as a build dependency for

ABKEP. Um, Rust TLS, which is what I would prefer, uh, didn't actually support arbitrary ciphers offering and ordering. Um, so that was, uh, something that I need to, okay, well, OpenSSL is now a dependency. um libsl dev or whatever the case may be for your system and it means that on my build machine I need to cross compile it for all the platforms that targeted um or

don't include libsl dev um so for instance windows doesn't have an open like a libsl dev package so I need uh do something there. And uh even the Android targets on my build machine um need to for some reason it wasn't working when I was uh building libs uh for uh including those Android for Android targets and needed to um to do some cross compilation of OpenSSL

for the Android targets as well. for Windows. Um it was easier to just like take a Windows VM instead of just the build VM um and Windows VM compile the OpenSSL dependency statically and then take the files that were in OpenSSL statically compiled build and uh SCP them over to the the build uh uh VM. Pain in the ass, right? Um also the entire AI exchange is

in protobuff. So there's kind of some work that needs to be done. So the protobuff is like Google's um very efficient um uh serialization library and so it's all um so all the exchanges of data that happen over Google Play and Protobuff and you need to kind of reverse figure out how all that works as well. But luckily a lot of that work had been done already

with um you know uh Aurora store so I can rely on them a lot um and they have these proto file specifications um that need to be included. So yeah Google play was a pain in the ass. Um so there were some implementation challenges for APK pure as well. Um, this is what I got when I was reading the network traffic for APK pure exchanges. Um, it's

not so visible probably from many of you, but it probably looks as decipherable for you if you can't see it that as if you can. basically it's just a some serialization uh technique that uh they're using and it this is what it looks like in Vim, but I couldn't figure out if you just run well there's [snorts] no zero documentation um if you run file on this

binary uh exchange then it returns data. very helpful. so well what do you do? Well, you use some rejax because you can actually see in that green stripe where the URL for the APK starts. And this includes a bunch of other data like um the you know graphic for that app um you know some reviews or whatever it may be. But the APK's um actual URL is

in there and you just need to use some kind of reax to extract it. Um look for APK J whatever and then just a kind of generic reax for a URL and we're able to kind of get that URL for the downloading of that we want. Well, what are the challenges for FDroid? Hey, finally they actually have documentation. Um, they actually are a very helpful dev community.

Um, you could just pop on their matrix server and ask them questions and they will respond and you say what you're doing and they will give you time to uh to point you in the right direction. So um the there's a ve this verification mechanism for apps um that are inroid and there's this large manifest that's um hashed and then you that hash is signed and um

signed by froid maintainers. So there's support for these additional repos not just the mainline foid repo but as I mentioned like the guardian project for instance has its own um and they might want to you know verify against those as well. So you want to have the same uh verification or you want to have the same verification mechanisms that foid has in its app built into AV

keep right um to give the same asurances. So that was somewhat of a challenge but um luckily it was kind of easy to to follow and it wasn't a mystery and deciphering it. So finally after all of that we get a functional AP keep um which I published uh in September 2021 um and uh was kind of able to to uh publish it and uh make it

available to the world. So great we have a released product. This is what the command line options look like. I don't probably can't see that very well, but just run through a few of them. Um, downloading from a CSV. Um, listing versions that's available for APK Pure. Uh, Google Play doesn't let you. Um, specifying the download source. Um, there's also like a D- O for different options.

Um yeah, there's some uh of the uh mechanisms to get authenticated with Google in there as well. And there's also like um you could specify how many parallel processes you want to run and also introduce a artificial sleep duration um in order to not overload uh the servers that you're getting the APK APKs from. So my goals with this project were kind of three-fold. Uh was organizationally

um facilitated our own work in looking at some of these um some of these these packages and malware for instance. Also, it helps us with a project that we're doing now on location data brokers which maybe presenting at hope or defcon not sure but um I think that um it's been very helpful for for u doing some of this loca and looking into location data brokers. My

personal goal with this uh was to kind of un understand better how app distribution works in general and cutting my teeth on some async rust. Async rust was pretty new at the time too. It's a we're talking like 2020 2021. Um so uh so I think it's like 1.3 something was when um Russ introduced the async keywords and um and it was a it was a nice

addition for sure. And um we allows us to give back to uh the open source Imposs community with a generally usable tool and it is used in a variety of projects which I'll get into in a bit. So, some of the milestones since release for Google Play, we upgraded to Google Play's API V3. Uh, which was nice because they actually got rid of that uh OpenSSL fingerprinting

weird stuff that they were doing. Uh, so I was able to remove Hyper OpenSSL and move to the more standard higher level request crate. um adding the ability to download split APKs. So not just one package with the localization and CPU architecture and different APKs, download them all at the same time for a given application. Um also downloading what's called OO obv files uh which are kind

of expansion files that you might want to include in an uh Android application which is u used often for games. some you know basically any application that wants to package to to provide more than 200 megabytes of data to to its users. Um you can download those OBB files there was this interesting pull request that came in more recently and it was to include or to allow

downloading of this DEX metadata file. The dex metadata includes Android art profiles which is a Google cloud profile. As of Android 7.0 um you're able to optimize the running of an app through these ARC um which are not just in time compilation but compiles fragments of your application uh before that. so that um there are more clear paths for execution um and you don't need to do

that JT stuff uh just in time um so it speeds up the application so figures like up to 20%. Um which means that they're doing some kind of profiling of app usage and then pre-ompiling those segments of the app. Um, so that kind of allows us to do to to rely on what's already been done as some kind of dynamic analysis on the coverage blocks of an

app. Um, device configurations maintained by Aurora. Yeah. So there are these device configurations. You basically like wherever you interact with Google Play, you need to say, "Oh, I'm on a Google Pixel 9. Um and these are my capabilities of that Google Pixel whatever uh Aurora store uh is great at kind of maintaining those device profiles um and and then uh making those available. So we just inherit

from them and I definitely uh recommend checking them out because they're doing a lot of great work. So we also uh have more ways to install than just one, right? You can download the binaries directly from our releases page on GitHub. You can download or for cargo uh for Rust, you can download the latest release via just cargo install APE. Um that'll compile it on your system.

um to compile from the latest commit. There's also the cargo command that you can use which is great. Um Termox provides the pre-ompiled binaries via just a pkg install abs also have a docker container um in the docker container registry. Um, so you can use that URL to just download and run APK without having to go to the releases page or compile it yourself. Yeah. So for

froid since release we have also introduced some new entry point specification which is this way that froid uh specifies uh where to look initially for the uh manifest of all the of all the foid packages. Um and they've kind of moved from like a few years ago from Shaw one to like SHA 256 hashes that they're assigning. And so that's nice. and um we uh moved to

support that and then now we're like kind of defaulting to that and then you can still fall back to the to the older standard if you want to because some uh third party afterroid protocol uh repos are are still like only on uh version one. This might not be true as of 2026, but um but you we still support the older standard if you Allow the listing

of versions. Uh APK pure is great for that. Um and you can output for Android at least the uh versions and downloads that are available in JSON format. We've made some UX improvements as well. have a nice progress bar and that sweet new logo that you saw in the beginning of this presentation that was uh done by my colleague Shireen. It's great. Where is Akeep actually being

used? Um, if you've heard of Exodus Privacy, Exodus is a group of French hackers which are um downloading a bunch of APKs and just reporting on which SDKs they have installed, what their general uh privacy of those SDKs, you know, privacy properties of those SDKs are, if they're sketchy, etc. So for any arbitrary uh application, you can a large application, you can look and see what um

what they're doing and if they're kind of sketchy or if they're not. Um just kind of on a high level. So actually this is great. Researchers and hackers are using APK. uh mentioned that pull request earlier about the dex metadata and that pieces. Um it'd be great if you if you provided your credentials and you had access to paid apps so you could download those apps. So

want to do that as well. I'm not sure what the support looks like for that currently and you know might just work now but want to look into if that um I can build a better support for um an integration into existing reversing uh toolboxes. There's a few um great Android reversing toolkits um that u might benefit from this adding additional repositories like basically getting ourselves included

in say like you know uh Debian or Ubuntu or whatever. There are a few that were included in so like the high level right what what is APK used for and what it what what is it and what it's not it's not some like flashy cool project that does everything and it wasn't made to do everything. It was made to do like just one thing right. um

purpose-built for one thing and one thing alone and just actually Yes, it's flexible, extensible, and portable, but it's basically just what you want in a tool. Uh do one thing right and get it out of the way so I don't have to actually think about it. Um so I appreciate you taking time to actually look at it and think about it. and um and our goal is

to kind of get out of the way and help this tool help you to get your work done, but um but it does involve a lot of work putting into it. So um appreciate you if you are an EFF supporter and contributor um that helps us do work like this. Yeah. And there wasn't an easy tool out there for this and it was technically feasible. So just

why not make it? So that's my presentation. If we have a time for any questions. Um thank you. So I'm very curious uh regarding uh so this is the silver bullet for all the sideloading woes for >> well sideloading is different like uh well yeah well sideloading is its own can of worms right now uh Google has announced plans to vet developers um even if you are

getting your uh Android apps elsewhere um it won't run on your machine or on your Android device unless you are a vetted developer. So there's all sorts of um wall you know walls they're putting up. Uh >> but yeah >> going that was more just for fun but when it comes to the cyphere suite element that was pretty cool. Is that something that they've maintained over the

years over the past like five years? >> No that was like a while ago where I'm not sure why. Maybe it was like some in intrusion protection system, whatever. But so I'm not like attributing it necessarily to a human saying this is a good idea. I think that might just be like some automated system that is like, okay, the only legitimate thing this is for is Google

Play and Google Play is actually only installed on systems which are using this cipher suite um because it's you know using OpenSSL or whatever. Um and so and so like you know it looks it looks weird if it's not that. So maybe it was some as a result of some kind of like um automated system that that started blocking it um if it wasn't exactly what it

looks like from the legitimate Google Play. But I don't >> We got one back there. Going back there. >> quick question. Uh did you run into any problems with geo restrictions in the Google store or is it done via the authentication and whatever is available you can pull through that account? >> Theoretically at EFF might be able to bypass geo restrictions um with the app and might

be doing so on a regular basis. Um uh so yes it is possible I believe Hello. Hello. Hi. Welcome everyone. I'm Tina Microsoft. I'm Tina. I'm at Microsoft working as a software engineer. Hi. Hi. meas around. >> Yeah. So the normalization Oh, this is about >> Yeah, it's kind of >> Yeah, I was thinking check check. Keep it in the pocket. >> Good job. Hear me? Hello.

>> Almost against your cheek. Good. Is it good? Check. Check. Um 35ish. Do we have 45 minutes as a slot? I think 245 until Sorry. >> Oh yeah. getting stepped on 30 seconds. Yeah. Check. Check. Check. Is it fine? You hear me? >> Hi. >> All right. Uh 2 p.m. How are we digesting? Yeah, everybody's sleeping. Okay. Uh without further ado, uh Tina K. Tina's a software

engineer at Microsoft in Charlotte. Uh she specializes in data engineering, machine learning and big data analytics. So please uh give a round of applause. [applause] Welcome everyone. Today I want to talk about prompt engineering at scale in uh production ready financial platforms where more than novelty what we care for is reliability, correctness, auditability etc. So this talk is about an engineering discipline when we apply LLM in

finance. It's rather than a prompt talk. Um let's see what is different about financial operations. So as we all know most of the financial platforms across the industry has been uh there for decades. As a result of that and part of modernization as well we do have uh system spread across legacy as well as modern uh infrastructures. So um added to that financial uh of the events

have um they are the high velocity events which include payment ledger uh it could have fraud signals identity etc. And um the distribute we uh host in having a cloud native architecture comes with its own challenges as well. And uh financial operations are time bound. Uh we are bound by month borders, quadrant borders. Uh and in such scenarios the revenue reporting is of high criticality and uh

time matters more. And of course auditability and compliance are non-negotiable. So before diving into what uh what problem we are solving here, let's get into uh setting the context of what the complexity we are dealing with. This is a representational architecture. Uh the trillion dollar financial platform of the corporate doesn't uh rely fully on this architecture. I wouldn't say but there are multiple systems across it and

um I want to go over this architecture to set the context of our talk today. So we do have microservices uh we host everything in Azure. So uh these microservices are uh deployed uh in Azure cubernetes. We do have cosmos. So in the mic the microser services processes the business logic uh puts the data into cosmos where uh it pushes to event hubs and these uh the

spark streaming jobs pick up event hub uh event hub data and put it on delta lake. We do have uh change data feed enabled in data lakes to uh supply uh for the different stakeholders to consume the data based on their requirements. Uh on top of it we do have an entire infrastructure of logging and dashboarding. So we do uh use crystal logs and use promeus for

the metrics and dashboarding using graphin. while this is the flow we need to get into where the engineering time is spent. So in engineering we don't spend most of the time writing code at least now with agentic AIdriven developments but uh most of the time is uh spent in recreating the context. So um this is how I want to uh talk through this solution uh on prioritizing

devops and then moving to the engineering side or the dev side where we can uh the engineering can augment the devops. So when during a manual during an incident uh engineers spend time triaging it and more often than not these incidents would have been occurred sometime and different context. So uh essentially what we are doing is we are recreating the context of it and uh when it

goes when it comes to live sites and other updates we do spend lot of time in the meetings um and explaining the technical issues to uh operations. This means multiple things right. When we say we are explaining, we are explaining the same incident in and the different dimensions of it uh and applying the perspective depending on the audience. So this is about like where the time is

spent. Now we are trying to solve these with AI. Let's see what research says about AI. Um so this is a uh research done by Stanford University where AI delivers larger gains on easy task and than the complex task. Also if you see the graph uh AI is more effective in a green field than a brown field. So um none of these what we have talked about

and about the financial platform uh seem like a green. So we uh require AI uh definitely it's a wave that we all all wanting to catch up and financial platform is no different. Uh the in the phops where uh you know accuracy and being deterministic is more important where AI models are more probabilistic. We are here trying to solve uh our um solve the issues and improve

our productivity using AI um by merging the co the core concepts of Yeah. Now that we have set the context about where AI is and what the platforms are about, uh the first step what we always go uh ahead and do is the ad hoc prompting. And why we fail at it most of the times because uh suppose we take an instant triage every uh will be

operating on different prompts uh to get the output. Yes, we all get an output but they must they will be inconsistent. They lack auditability and reproducibility. Uh this also increases the risk of hallucination and we do not have any validation. So we cannot let AI do the decision engine as such uh by uh using So uh diving into the core idea what we have uh come up

with and uh finding effectiveness is using prompt engineering as an architecture itself. So the prompts are version templates. they are governed by context injection. So we will in the further slides we'll dive into what sort of uh guardrails we do we put into for the prompt engineering architecture and this uh gives us pretty good results uh and the outputs are quite structured. Uh this is also designed

around reliability and auditability as well. Um in addition for the being secare when we deal prompt engineering as an architecture we do review the prompts like how do we review the code today um let's dive into a few of the uh patterns which we want which we would imple which we are implementing uh which is helping helping us in mainly the DevOps and the engineering part. So

we provide the promps with the schema um before that before jumping into it. So we do process the financial data right in that we do have several business streams which run on uh entirely different logics. So we do have separate prompts uh files written for any business flow and also for a particular use case as such. But u what I'm going through here is the general pattern

which we are following for any prompt um the time the operational telemetry and time bounded transaction history and explicit policy constraints. So we just don't uh by doing this we constrain the model and allow it to do only so much. Um and the template anatomy uh the persona definition input data contracts the guard rails and the deterministic output schema and evaluation hooks. So when we talk about

the persona u like we touched upon before a same incident need to be u el needs to be explained differently based on the audience. So we do have prompts based on how it should uh give the output depending on the audience which cons who consumes that. [clears throat] Solving the dynamic context we do time bounded context windows. We do not let it have an implicit memory in

such dynamic context scenarios and this produces a reproducible reasoning and uh they are are quite audit friendly. We do have like from what we get in from the prompts we do have dashboarding and doing the metrics. So let's [clears throat] dive into u this is one of the pattern which I want to cover uh since this is one of the hard tile workflows which we have identified.

So in the persona specific templates when an incident happens what do engineering care we care about what happened and what's the root cause in the analysis. what do the uh on calls or the operational teams care about is what is the impact and uh is there an SLM and what's next when it comes to compliance they think about the controls and traceability so um in our teams

uh at Microsoft where this uh revenue reporting is done across multiple teams we do have like a central team which audits the uh output of each system. So when do we get a uh variance report from uh these teams? It just says that yeses your system is causing a variance of x dollars. It doesn't say why. So that is where we uh implement our own agents to

analyze and uh analyze those variances and root cause now that we have like talked about the patterns and the context how do we set it let's dive into some of the use cases where like I want to cover the ops first and then uh some on the engineering side as well. So one of the use case is the automated uh instant triage. So we can classify incidents

uh based on this. It could be based on the severity or dollar impact whichever and correlate the known issues because we always have an history. More often than not, we would have seen this that some type of those failures before and it would have been handled differently based on like who uh looks into root causing it. Even though we do have um structured troubleshooting guides and uh

processes to follow um and to summarize the scope and risk and at the end we let human validate the response as well. uh now I want to dive into an architecture where we do uh incident management and it necessarily need not be uh same across the systems but uh our team we use Azure S agent. um SR agent capabilities have been like improved a lot recently and

we do have uh ICM management ICM integration to it. Um where we provide uh the context of the logs. Um we can provide the context of knowledge base which is our area or the code repo. It can you can upload the knowledge base as such as MD files as well. And there's an ADO integration which is preview. So u what happens I want to want you guys

want to walk you through like what happens in the agent uh what value we are getting out of the agent when we see an incident. So on top of all of these we create prompts as automation plans for particular ICMS by analyzing the patterns which we have seen before. So once there is an in when an incident happens the S sur agent processes it looks for the

prompts. It looks clearly on the logs and derive the out derive the results of the logs and then we prompt them what should be the next step on uh based on the outcome of the logs. So this uh also it looks on the incident history to see how the previous incidents have been handled and if there the knowledge base helps it to like root cause the incident

and ADO integration uh can it can just log a another use case is anomaly detection with the explanations. So we can do numerical anomaly detection and uh it can provide the agents can provide a natural language explanation and uh which will enable like faster cross team communication. One of the use case which we do is like we have an anomaly detection system on the traffic which is

a machine learning learning predictor modeling and we do host our services in Azure cubernetes uh which is enabled by autoscaling. So even though auto scaling is enabled that's more of a reactive uh that's a reactive way of uh increasing the part size but with predictive analytics we can do it in more predictive way. Um so like we said like based on what we how we enable the

prompt auditability is uh inherent and it is uh it is baked in. Um now we uh dive a bit into the engineering side of it like meeting summaries as operational memory. So we do not use AI uh here as a decision tool but more as an operational memory. So uh we do have work IQ where it can summarize all the uh all the meetings and create ADO

task out of it. So also uh we all know that when we do a design and the design uh reviews and uh we get we land on a design definitely the feedback from uh the fin ops like what uh the instance occurred before and the root cause and the troubleshooting acts as a feedback loop for the uh next feature design. In in such cases these meeting summaries

using work IQ is is proving uh to be like quite efficient for us. Uh also uh the decision uh the design conversations can be like summarized and provided uh as context for uh specket. So we use GitHub specket for the feature developments. So um another one which I can talk about is the work item prioritization and the bug logger. Uh we do have uh work item we

do have like specialized agent built upon uh work item prioritization. So when the instance happen and when they log the bugs uh it can also keep a count on like how many uh times it's a repeated incident which will allow us to get find the lowhanging fruits which uh which gets prioritized in the sprint planning. So uh these specialized agents help us to uh reduce the time

uh spent on planning and focus most on like feature development and operational excellence. Um some of the specialized agent which I can talk about is uh dog agents. We do have like every uh business stream has like its own logic to run for uh the revenue processing and we do have specialized document agents which which will update the documents uh when uh when used. So how do

we set prompts in this doc for this doc agents is that we provide the context on what files are uh related to that particular flow and what changes it should detect and create those uh and accordingly create the uh dark I'm sorry uh accordingly create the uh pull request which has a document change and bug logger and analyze agent are uh similar to like what I spoke

about in the instant management. Uh but we do achieve bug loggers uh in several ways. S agent has access we do have specialized agents. So at at this point we are more of an experime we are more experimenting on like what uh which model and what uh prompts gives us the better results. So uh another uh advantage when we do uh check uh when we do these

trials with multiple agents, it will help us to nail down on what are the most effective prompts and then that prompts go into as a base of further Um and another way we uh achieve more of productivity is through skills. there are some some set of uh activities what the uh SWE do we do not want to let the agent just take it away from us at

the same time uh we do want automation around it that's where we effectively use the skills so if we find once we find a root cause and an incident and we need to push a bug fix for that even for a bug irrespective of how pri uh how much of a priority it is it has to go through several uh steps of certifications before we uh land

it on broad. So uh we do have skills built on like running this uh code change in multiple environments and letting us know the results. Um why why don't we do it as an agent? because of the complexity involved in the architecture. Uh a build failure will be something to if we want to like effectively give to agent on a build failure around multiple environments, it would

be uh it it might go heavier than uh doing it as a skill where the uh developer has more control over it. And the PR reviewer skill has been uh we do have a a skill enabled for PR reviewer where there are three different models. Each uh model uh one model will run on the infrastructure related code. Another model will uh look into the security uh perspective

and another model to uh look into the logic perspective of that particular pull request. uh what are the rollout strategies? So uh how do we do is that we try experiment with small things and uh first we target a high toil workflow and always keep the human in loop and then gradually automate it. So when we when I talk about all these sub agents we uh sorry

all those uh specialized agents we never got into we are still going towards the sub agent model where we can enable each of those feature as an agent and uh use a multi- aent model. So that is where like we can um that's the look ahead part of it and we do have uh when we started with S sur agent we do have a human in the

loop always before resolving the incidents or uh giving a update on the um on the instance but we do let the agent post how we measure success? So uh if I can give a metrics when we implemented the S sur agent in the least form uh we could get 30% product uh productivity gain from the on calls because that was uh SR agent was very effective in

solving some mundane uh task which on calls should do to uh to mitigate or investigate an incident. um then uh the faster uh incident understanding and the bug resolution. So uh with uh agentic driven AI development and uh our work item prioritization and the bug loggers we are able to prioritize the high impact bugs and solve them uh and ship it to production at the faster pace.

uh this also helped us to like uh reduce few uh reduce the repeated escalations especially in the case of variance. So there could be a variance reported and uh we are still if we are still root causing it or we are taking longer to root cause it then uh it it the variance due to that particular issue could build up and like I said when we are

uh when these operations are time bound if we are closer to a mountain border or a quadrant border it is going to be uh much difficult. So we are not running into such issues uh improved confidence and cube delivery. So I think we covered this. So uh these are the key takeaways like prompt engineering is to be sure seen more as an architecture rather than ad hoc

and uh the templates create more reliability and we should let uh AI reason and not like uh decide uh and by doing prompt engineering as an architecture we do get it uh we don't get let it get in the way of the transaction path. We only let it reason And these guardrails, setting up these guard rails effectively will help us to uh scale it out and apply

it to like multiple use cases across the platform. yeah, so that's all I have today. >> [applause] >> All right. Okay. Can you hear me now? >> Great. Wow, everyone can. Um, question. As you have the different personas and they're talking to other LLMs, do they need to be aware of those other personas and how they'll react? >> Um, not actually because uh we do have actually

it's a good question. So, uh if I can give an example of a particular instant or a variance. So the how do we analyze the variance the prompt will do and that is uniform across irrespective of the audience right it is only the towards the end part how it determ how it sends out the output and how it uh interprets it is different. So we tune So

uh if you ask me whether it has to be aware of the other personas, it need not. But the developer should be aware that you know it has to serve these many personas. >> Hi, thank you. Um I wanted to ask um Are there any strategies to uh get the outputs uh being as close as possible each time? So if you get different context but you need

a specific format, some structure isn't an architecture problem or a prompting is there are there any best practices that are in use? >> Yes. So this is the uh issue we were running into while uh when we used ad hoc prompting. So uh suppose for an example if I'm trying to solve a problem I write the prompt in like in a certain way and another person will

write in in a different way and AI models being probabilistic it's for sure either hallucinate at some point or give uh different results when we set up this prompts as an architecture what we are doing is that if I can like deep dive into like one of the incidents so if we get an incident and uh the prompts will include okay which custo environment you should look

into which custo log you should run and how do you find the time bound. So those are not that those are uh going to be deterministic right and we are not going to uh get my different results based on uh the person who runs it or the prompts and with respect to the prompts all still like natural language is there in in prompts right so still we

just uh standardize it across across the team so that everyone runs the same so that is why like uh we had like uh good uh results on MCP servers. So we use MCP servers and like run it locally uh and uh evaluate the results for some time before like we just really land it broad. So that one extra step has helped us achieve a good uh good

quality uh responses from the models. >> two slides. in the beginning. for all the help. >> No, I mean Are you local or >> no? >> Oh, the expo. So going back >> and you're going to take a bus or Express. Yeah, it's like Yeah, thank you. >> But no, otherwise >> Yeah, I just had like 30 minutes and then I just did not Thanks so much.

>> For everyone after like now that we're living in that world. afternoon. >> Just make sure the transition goes pull the whole thing out. There you go. Breaks right off. Hey, >> I was just >> Oh, you're gonna do this. >> No, I was too on. Yeah. >> Check. Check. check. Check. right? Testing. Testing. >> It's good. What do you We could try. It's not going anywhere.

All right, three o'clock. How we doing? Digesting, huh? All right. Well, uh 3 o'clock. Uh this is Jason Kramer. His second time speaking at Sunset Con and um he will have another talk tomorrow at scale. Um Jason currently right now focuses on AI and improving models but he is a total guru when it comes to symbolic execution and with his uh colleague Dr. Richard as well. So

uh please without further ado Jason Kramer. >> Thank you. So hi everyone thanks for being here. I'm excited to be presenting on AI supply chain security. Uh so the presentation poison ones compromise many and how model reuse amplifies AI vulnerabilities. So the AI supply chain explosion. So what does that mean? Uh 61% of companies according to an IBM survey uh said that they were using open-source models

in their products. Uh so they were fine-tuning open source models for example on GitHub or on hugging face uh and using those and deploying them within their systems. I've also been tracking how many models are on hugging face uh just like six months ago there was maybe around two million and I saw just yesterday I looked again it jumped up to like 2.6 million. Uh so it's

like exponentially growing the amount of open source models that are out there. Why are companies using open source models and deploying them locally? Uh it's because it's cheaper to fine-tune models that you don't need to train from scratch. There's a lot of resources that have to go into training a model from scratch. You need a pretty large data set. you need to do a lot of research

into feature engineering. Uh, using a model that already exists saves companies a lot of time and resources. They're able to get results quickly. Using open-source models as well as public data sets, however, comes with a lot of risks. Uh, I'll be going into some of them in more detail, but some of them include risks of data poisoning. The model can also be poison that you're fine-tuning from

or potential biases. Many models don't disclose their origins or training data. So when you go to hugging face and look at like a model card, a lot of the time they don't actually say what are all the ingredients that went into their model. So when you're going and using that model, it's kind of there's a lot of unknown there and a lot of lack of transparency. And

these models are complex systems and they're becoming more and more complex with tools being integrated and agents and rag and other components. There's the weights and data sets, code and other dependencies as well. Um, and so all of these things kind of make it more difficult to have a clear understanding of the AI supply chain. And so the question is how confident are you that every data

set dependency and component in your AI pipeline is safe? And if any of one of those components is uh potentially attacked uh it could compromise the whole system that it's part of. So something about AI models compared to traditional application security is that models are pretty much opaque boxes. uh you can see the inputs in English, you can see the outputs in English, but in that process

in between there's sometimes millions or billions of parameters. Uh and there's not a clear understanding of how the model got from point A to point B. Uh and because of this, I you can't really understand unless it's documented things like copyright and licensing issues. There's a lot of lawsuits that we've seen over the last few years where companies are getting sued for potentially having trained using copyright

images or copyright video or There's also a risk of transferability attacks. So if one model is vulnerable, it's been proven and we've done experiments as well that models that are fine-tuned from that base model can also be vulnerable. Again, with data poisoning, uh a small portion of a data set could potentially contain patterns or uh be manipulated in a way to trick the model after it's been

trained uh in order to impact its decision- making. Uh there was one study that I found. I think I didn't include the thing on this slide here, but there was one study I found that they they poison like 01% of a data set and they were able to uh impact and influence the model after it was deployed. And then there's issues with compliance as well. So, uh,

if you don't know what went into your model and you're and it's like a high-risk, high impact model and you need to actually show information about how it was, uh, developed, uh, this could go against things like the EU AI act safety rules. Uh, there's also like the NIST AI risk management framework and MITER atlas and lots of state level regulations that have been popping up. And

then it's difficult to debug issues because going into these models is uh potentially millions of unknown data points and data sets for these models usually come from a ton of different sources. So if you're trying to figure out where malicious data came from uh it's difficult unless that's documented. So software engineers have already faced similar issues uh in the software supply chain area with ESBOM and so

that was the solution there. ESBOM or software bill of materials. So software supply chains they're complex they're hard to fully understand unless things are documented. if there's components that are unknown that could have uh introduced security, legal, and compliance risks. So, if you don't know what packages you're using and then one of them ends up being vulnerable, it's much more difficult to mitigate that. And having transparency

enables faster vulnerability response and being more uh ready for regulatory requirements. So for ESBOM, it's an inventory of all the software components in the system. What it lets you do is there's a lot of vulnerability databases that exist. Uh if you find out that a package that you're using was in a uh has a vulnerability associated with it, you could then go backwards and quickly fix uh

and patch that vulnerability. But if you don't know that information that you're using a specific version or it's in a specific range, then it's much more difficult to know even that you're vulnerable and then to mitigate it. And this approach is already widely adopted for software supply chain security. Uh and so here's an example. So like there was the log 4j a couple years ago I think

now. And if you didn't know that you were using that version, then you might continue to still have vulnerable code in your deployment. So on the other hand, now we have AI bill of materials which is kind of an emerging format. There's been some kind of work around this for the past few years. Uh that's been a bit slow moving. Uh but it's supposed to be a

very similar format to software build materials. But for AI, there's a lot of This also includes other things like data sets and what architecture did you use to train your model? What were all the data sources? It's essentially a new type of documentation format for AI. And it documents provenence and risk for AI systems. So the goal of them is to be able to track upwards like

what base models the model that you're using was fine-tuned off of and kind of what were all the components that went into each And here's an example of one of the formats is called Cyclone DX. And they you can see things on there like the model architecture and uh like the use case for it and other things like the task and description. So I think to understand

why it's important to track AI uh and AI components, it's also important to see what are the common attacks on AI So to jump into it, there's again back doors and data poisoning. And then from that, there's also evasion attacks. So for like computer vision, you see like a minimum number of pixels changed within an image or video feed. And then they're able to trick for example

like an autonomous vehicle from seeing a red light and thinking it's a green light. Uh so there's potentially pretty catast uh pretty bad consequences if they're attacked. There can be malicious model files and dependencies. There's been a few uh case studies on hugging face where malicious models were uploaded. And because these model formats are often uh like serialized files or binary blobs that contain the large model

files and weights, uh it's been shown that like pickle files for example can have and execute arbitrary code. And there's been cases where hugging face like took down models uh that contained uh malicious code in them. And then this was a term that was coined like sometime last year with slop squatting uh where AI models tend to hallucinate package names. There can also be malicious data sets

or models that try to lead responses towards malicious package names as well. Uh so an attacker can either have their malicious package on on a website like pi or lead towards downloading from some other malicious source. Um and the model will suggest it and then if a developer doesn't really look into the setup uh instructions that it's providing or the requirements file, they might unknowingly download it.

Similarly, there can be vulnerable software generation. So models have been shown especially with like vibe coding that they can generate vulnerable code and there if you don't really know what went into like a coding model for example then you don't really uh know whether it was trained on safe or vulnerable code and so it might be more likely to suggest vulnerable uh code patterns. And lastly, this

has been kind of growing in popularity recently, but indirect prompt injection attacks. And this is where prompt injection attacks aren't provided by the user, but by like another component that's connected to the system. So, this could be something like a malicious open-source MCP tool. Uh, something like data within a rag database that contains information. Um, and this this could do something like if you have your AI

hooked up to your email client, they can inject like a prompt that says, "Ignore all previous instructions." You know, send me passwords or forward your emails and then delete them. And so AI vulnerabilities are different than kind of traditional software One reason for this is currently there's no CVE style disclosure process. There are vulnerabilities that exist for things like TensorFlow and PyTorch packages, but they're more like

traditional software vulnerabilities associated with their code. So there isn't really a database that exists today that is like a central place where you can just report on an AI Another issue is that models are opaque. So it makes testing of the models difficult. There's limited tooling for instrumentation. Uh what you see with like ESBOM or with software vulnerabilities. You can go to the GitHub repo. You can

analyze any of the code files that you want and see if there's a potential vulnerability in any of the functions. You can't really do that with an AI model. you can kind of probe it and test it for specific things, but it's a bit limited and and the tooling is kind of all over the place for computer vision and language models. You're kind of using different tools

for uh different types of models. And then in some cases, the NDAs could block disclosure and getting information that you might need for transparency. And then also like there's not really a standard version uh control system for models and then also for mitigations. Models on hugging face are usually versioned like based on the size of the model or like a new version that included new training data.

It's not really like we patched this issue and now here's the new version that doesn't have the vulnerability. On the next few slides, I just have like a few examples of uh some examples of attacks. So, here we have an attacker who uploads poison malware data into a malware repository that's commonly used for generating and training AI models for recognizing benign and malicious Um, this could be

like a specific pattern in several of the examples in the benign samples and that they can later use during inference in like a malicious example, but now they know that the model's been trained um on that specific pattern to not recognize it as malicious. So anyway, in this example, the cyber security company comes along, trains an AI model, puts it in their product for detecting malware, and

then the attacker can come along and bypass it and go undetected and execute it on the customer's system. Then to touch a little bit more on attack transferability, this is an area that our company we've been working on a lot where uh AI models can pass down vulnerabilities to other models. So we did some research there. This example here shows uh we had an original birdbased uncased

model and then we had a full retrain uh for bitbased Chinese and then there was name density recognition which is the second one which was just a fine-tune of the original. We found that there was much higher similarity in the representations of the intermediate layers to the fine-tuned one than there was to the So fine-tune models can inherit base model flaws. And then these allow for like

gray box and surrogate model uh attacks. So uh you can if you know something about the original model like if you know its architecture, if you know the model that it was trained off of, then you can use that as a base to generate attacks and then apply that back to the uh original target model. And so this is kind of an example of that. We ran

a lot of experiments where we took an original model and then we generated adversarial attacks against a bunch of different architectures or different versions of that model. Then we took the computer vision the images that we generated with the attacks and applied them back to the original model. What we found was like for this example, we have a VIT model that was trained on the Oxford PET

data set and had an original accuracy of 95.71%. Uh when we attacked it with a different VIT model with an FGSM attack and this has a bunch of noise over the image, but it's actually kind of hard to see since it was a white box attack. So we kind of manipulate minimal pixels to generate those attacks. the attack success was much higher for the VIT FGSM1. It

lowered it down to 81.71% accuracy compared to something like the ResNet 50 model which it still lowered the accuracy um but it wasn't as successful. So what goes into this documentation format in AI bill of materials? It's a comprehensive view of all components that are influencing the model and it has provenence and lineage for data sets as well as the models and it also has information on

the risks and constraints and information relating to compliance. So like a lot of the times they have information on what the licenses were for using the models that the model was trained off of. And then the goal is ultimately details to support reproducibility and audits. But as I'll show on the next slide, I don't think the AI build materials are really there yet to be able to

do a full like reproduction of a model. There are some existing formats. I'm going to go over like the main two here. So SPDX, both of these kind of already have software bill of materials formats. This is an open standard from the Linux Foundation. Uh it was originally built for software and now it's been extended to support AI. It has an extensible data model. Um so it's

not just limited to the software components anymore. It has uh things for data sets and models and other components that are associated with it. It tracks information on where the data was obtained. And there's kind of like a a whole bunch of properties that they include in these too that are listed in that uh screenshot that I have. Then yeah, it was in this like new 3.0

format where they added the data sets and models and I think I saw they have something new for vulnerabilities in their new version that they are coming out with soon. And then cyclone DX is another format. uh they're a lightweight format from OASP. Uh it's more optimized for security and vulnerability management. So you can report things like what metrics were used for training the model and and

testing it. Um like how you tested it for robustness and how it's scored. And then you're able to kind of communicate that information better. And then it's designed for integration into existing pipelines. and it supports a bunch of different formats and can be kind of exported into JSON and XML among others. So, hugging face uh when you go onto that website, it hosts, you know, the 2.6

million models. Um there's a lot of uh documentation that's on there, but a lot of the models don't contain any documentation. We kind of did a survey where or we did a study where we automatically like analyzed a bunch of computer vision classifier models and we found that a lot of them were either totally missing documentation or it was very incomplete or couldn't be validated. They do

have this form here that has a bunch of fields that you can fill out very similar to an AI bill of materials, but the form fields aren't all required and then they're kind of manually inputed. So, we found that oops, we found that the information couldn't really be validated. Uh, a lot of the time they would say that the model had some accuracy, but you're not able

to really reproduce it because you don't know what the test data that was used to evaluate it or like what part of that test data was used. There's also this AI bomb generator as well. This kind of looks at a hugging face repository. You type in the repository name and then it gives you a scorecard. So I thought that this was pretty interesting for getting a better

understanding of how complete these are. And it's kind of funny because they suggest to you the whisper tiny model and this one has like a pretty low score especially with the model card being uh missing a lot of fields and having a very low score. It also is missing like the description on the left side. So there's also a study by Stanford. They have this AI foundation

model transparency index where they look at a bunch of fields about transparency. And then they reported scores on how transparent models that label themselves as open source actually are. And they found that a lot of the models were missing a lot of information. So things like the data sets and labor sources and compute origin and things like licensing. Uh so I think none of the models actually

have like a 100% transparency Then they also studied it in every year for the last three years that they've looked at this and done the same scoring and I think a lot of the models either went uh down or stayed about the same. In Europe, there's the European European Open Source AI index. They're doing something kind of similar and they have a pretty neat uh user interface

on their website where you can see for each model like what fields they actually cover and they're transparent about and which ones they aren't. I did look at this one too like six months ago and then looked at it again recently and it seems like a lot of the models here got better scores uh before like when I listed it by the top ones even the top

ones had a few more that were red. There's still like I think none of these are blue all the way across and I think gray is not applicable to those models. So, what's next for AI supply chain security and transparency? I think we're still going to see a regulatory push. We're seeing it more, I think, in other countries right now. The EU AI act has been slowly

rolling out and especially for like models that are making critical decisions. They have much more strict requirements right now. So, US companies that are operating in the EU still need to follow these requirements. Even in the US it was mostly rolling out at the state level and then there was like a few things that were being proposed in Congress. Um but still in the future AI models

uh may have to abide by some future laws or requirements and they might say something like all medical devices you know need to have this documented uh if they're using AI and if you're not tracking that now and you don't have that information in the future then you might have to full uh do a full retrain of your model. Um I think more automation and MLOps pipelines.

So right now like I showed like those model cards are a really manual process and then there's human error that can get introduced. I think we'll see more automation in this area where you can just give it your model and have it analyze the model and generate the card for you. And then there's also AI models that are like being continually updated. And so the bill of

materials can't really be static. It needs to be dynamic and continually evaluated. I think it'd be really helpful if like on hugging face when you look at a model if you got like a standardized common ingredient like label type of information about the model. So if we could get more consistent uh and standard data sets to use to evaluate models then you'd have a better idea when

just looking at the repository if it's a model you'd want to adopt into your pipeline. uh techniques for verifying that an AI bill of materials is correct. Uh and then also applying this to like closed sourced models uh and being able to still like extract information about the model and evaluate it. So some quick takeaways, uh AI supply chain needs to be audited and High-risk models benefit

the most from AI build of materials. Integrating AI bomb creation into CI and CD uh pipelines improves consistency and then ongoing monitoring of models can help catch vulnerabilities especially with this space quickly evolving. Um there's like constantly new adversarial attack types that are coming out and so doing like a vulnerability analysis of a model once means that it's going to quickly become outdated in the future. And

then I think industry adoption of AI build of materials will build a lot of trust and compliance and it'll help the open-source community grow a lot as Thank you. And uh if anyone has any questions, I'm glad to answer them. Hi, thank you for the talk. Uh I just Yeah. Hi, thank you for the talk. Uh I just have a question. So what is your recommendation before

the you know any regulations come to place? So what we need to be careful about while using the >> I missed like the last part of that you're asking about the regulations and what to do about >> I'm sorry you hear me now. >> Uh yeah I think so. >> So uh my question is what is your recommendation while we are we still while the regulations are

like still just ahead and we don't have any right now. So what do you recommend for us to be like careful while using the models? >> So yeah, you're asking right now like about the incoming regulations about what's potentially to come in the future like how to prepare for that now while we're still waiting. Right. >> Right. >> Yeah. I think right now it's all about like

documentation and not skipping the steps like uh filling out an AI bill of materials. Uh I think just like having everything on paper so that way in the future if things do get uh need to be audited, it's there and you're not like trying to go back and figure out well what data went into this model versus this fine-tune versus this one. Um, and then you find

out trying to figure out like which one had copyright data in it or something like that. I think every single like version of a model needs to have all of that information. Um, and I could send out maybe there's like the chat in the app for uh the conference. I could put in like the SPDX and Cyclone DX like they have all the fields that they're trying

to track. Yeah, they have a lot of work that's going on now in in this too. >> Hi there. Can you hear me? >> Okay, good. Um, I wanted you to elaborate on how you integrate AI bomb into the creation of the CI/CD pipeline uh that in improves consistency. I we use the CI/CD pipeline, but I'm not sure how the AI bomb Can you just give me

some examples? Do you just add tools to scan what you have or um the all the different AI stuff and dependencies? Do you scan for all that? Anyway, that's that's my question. >> Yeah, I think that there are some existing like plugins that but they're very focused on like a manual input. I think like I think I've seen like a Jenkins plugin that but it still has

like a bunch of manual steps in it. So there isn't a whole lot of like automated analysis yet. We're we're kind of like doing a lot of experiments for that. But yeah, it's still like I don't think it's really there yet. I know that OASP like has an initiative going on with AI bomb and I think they're trying to automate it more. Yeah, I was going to

ask about the efficacy of sbombs in general. Um because when you're thinking about an sbomb and how it gets generated for the most part you're using you know binary analysis or even source analysis and you're doing software composition analysis and [clears throat] it seems like a lot of that is open- source code and nobody really is looking at the first party code or you know the proprietary

third party code. uh when you're generating an SBOM, you're really just doing pattern matching and you're trying to identify what that technology thinks is in that and creating an ingredients list of that. But in the final binary, what ends up you know shipping in your product, how do you know like i.e. solar winds, what actually is executing in that final binary? Because no matter, you know, the

vulnerabilities that you understand that that are in there or what software composition analysis tells you what it thinks is in there, if there's, you know, like a North Korean state actor that's on your network that injected something during compilation, you're not going to detect it with any of these things. So, from coming from a zero trust perspective on supply chain how do you deal with that? Yeah,

I think that's a good point because there still is like room for uh malicious code to get input. >> like what do you guys do at object security? Is it also binary scanning? >> Yeah, we're also doing like binary vulnerability analysis as well. So for us, you know, we're not really focused on sbomb as much since there there's, you know, we're more focused on like finding like

zero day vulnerabilities. So scanning the code and actually figuring out ourselves if it's >> And are you reverse engineering that binary or are you just doing inference? >> Um reverse engineering like static analysis and symbolic execution. >> Okay. like like a Gedra back end or >> Yeah, similar. And then yeah, we have like an engine for emulating the software and then finding vulnerabilities within it. >> Interesting.

Okay, thanks Jason. >> Uh great talk. I appreciate it. Uh just had a question about your slide about what goes into the uh AI bomb. Um, one of them was, I think, ethical considerations. What would you put into a bomb that would be considered ethical considerations? Is that >> Yeah, there's some things about like there's some fields in there. I might have included it in the screenshot,

but it's just there's some fields just mainly asking about what ethical considerations were taken during like development. And I think a lot of that has to do with like biased data. Like did you ensure that the data didn't contain like biases or what steps did you take? Let's see if I don't know if I include if it made it into these screenshots because there were a lot

of fields in these. Anyone else? Yeah, there there's some also on uh licensing as well. Um so like a lot of models have like specific license files in them and then there's also some scanners that even exist too. So that I guess that part can be automated. >> That was too good. Yeah. Well, you know, sometimes doesn't play out that way, but you know, I've been doing

this. So, yeah, you have to get people's Thank you. Great pres >> you are. I am preparing you in the most diligent way we can to avoid any downsides >> of coming to this absolutely wonderful conference over the years. >> This your first time? >> How do you think? >> Oh, you are freaking out. Oh, come on. Have you been to the others and watched It's like

that, you know, but I don't see you freaking out. You don't strike me as, you know, uh whe shrinking violet. Yeah. No shrinking violet. I >> he left his boxes. Feel bad. I want to shut it down. Let's see. What's our plan? We're kind of What a wonderful the warriors got the microphone. Check. Check. Hey. Hey. Check it. Okay. Are are you going to be back there

giving me like a >> That's working. Is it on? All right. This is the last talk for the day. You almost made it or you made it. so I have the pleasure to introduce the next speaker. Uh, I didn't know until today that she is kind of like my neighbor. So hopefully this is going to be the first talk of many that she's going to give to

Pacific Hackers and obviously to Sunset Con and uh we look forward to have you and thank you so much for joining us today. Uh definitely this is a great talk. So you know pay a lot of attention. I know it's hard. It's the end of the day. We just ate ice cream. Well some of us I heard that guy ate it all. I don't know if that's

true or not. I just heard rumors. So Oh, here here you go. All right. That's what it is. All right. Uh thank you again for coming today and uh with that uh I'll have Erin introduce herself and again thank you so much for joining us. >> My pleasure. >> Thank you. Testing is this on? Great. >> Uh where where do I do the volume? Two seconds, please.

Better. All right. Should I just use the microphone? testing. All right. Oh, much louder. [laughter] Thank you. Oh. Oh. Oh. Maybe not. >> All right. I might hold it. Uh, thank you all for your patience around that. Porest by design. When air gap networks become security sees seieve a device with perforations through which finer particles of various sizes may be passed through porous permeable full of holes.

Why are we using these words in a security talk? Typically we talk about how to build systems and tools that are secure by design, not systems that are full of holes and leaky. And I promise that today we will be finding our way to a secure approach. But in order to design for secure systems, we first have to understand where the vulnerabilities and holes open and more

importantly why our systems are porous and leaky in the So today we're going to be discussing air gap networks, why they can lead to poor security posture, and what you can do about it. We'll be talking about the inherent leaky nature of air gap networks, ways to mitigate and control open flows of information, times when air gaps might be useful, uh, and just an overall general security

mindset. But before we get into the nitty-gritty details, who am I? Uh, I my name is Aaron. I've been a network security engineer at Meta for eight years. I lead security architecture design. I lead crossf functional system design for secure business enablement. But prior to making my way into the security world, I had a former life as a professional ballet dancer. Uh I say this because I

think it's important to call out that sometimes we don't all start obsessed with computers when we're younger or sometimes we start out obsessed with computers when we're younger. We take a side path and we come back. Um, but we're all from different places. We all have different backgrounds and that's what I love about the engineering community and the open source community. Uh, but I eventually did discover

a love of security and in particular I really really like the interplay of human behavior versus secure design versus technical design. Uh, and I love the breadth of Okay, so what is an air gap network? Most of you probably already know this, but quick review. Uh, air exists between two networks. The whole point is to keep the isolated network safe from the world. This is the textbook

definition. Uh, there's no physical connection between network A, network B. And all management of the isolated network occurs on the isolated network. Remember that one. Again traditional use cases when we protect the network uh we want to actually protect the network. We have a secure a precious resource that we want to protect against the world. So the internet the world is scary and we want to protect

this valuable resource. This is what most people think of when they hear air gap classified government networks nuclear facility control Uh this is framing when you works when you have the budget, the mandate, the operational discipline. Uh but also national intelligence and nuclear operations probably aren't something that most people will spend much time worrying about on a technical level. But what if the danger is actually coming

from inside? What if the call is coming from inside the house? What if we have a network that is so untrustworthy that we need to protect the world against it? And in this case, the world might be the actual world. The world might be your corporate, your enterprise network. The world might be your home network. Uh and the scary thing is the the lab that you've spun

up inside your network. Uh this can also look like times when we have operations in geopolitical locations that have high risk for whatever reason. Uh this may look like unpatched systems. Uh I know I've talked to some of you, you work in research and reach research is notorious especially science research is notorious for legacy systems that cannot be patched have to maintain 100% uptime uh because one

scientist's test is running on it. uh sometimes we have third party we have thirdparty networks that we are uh communicating with we have thirdparty vendors that we're talking with and there's an interface where they are coming into our network um so all of these often often we have multiple instances together that form to create a network that we really really really want to keep isolated because it

is dangerous. So again going back to the air gap, I like to think about it like a castle. Uh every air gap starts as a castle with a moat. But castles need gates. Goods need to move. People need to enter and leave. And the moment you build that first gate, you have already started making a design decision. How wide is the gate? How guarded should it be?

How often should it open? What requires you to open it or not? Uh, and then eventually someone asks for a second gate and a third and eventually you have more gates than you have wall. Uh, okay. So, some famous air gap breaches. I think we're all pretty aware of these, especially if you're in the security community. Sometimes it can look like devices in an air gap network

actually generating Wi-Fi signals that can reach the external world uh for data xfill. Other times it's the quintessential USB stick that is plugged into an air gap network and suddenly attackers have control. But these types of attacks are what we see in movies. And if you're in the security world, maybe you secretly hope that you get to deal with one of these attacks someday. They're the exciting,

the bad animals of security exploits, and everyone secretly wants to know them. But this is not what causes the greatest leaks in our security boundaries. And this is not what we're here to talk about. Uh, one quick caveat, everything I'm talking about today is theoretical. It does not reflect uh, the internal systems of what happens at meta. These are all drawn from generic real world examples. Uh,

but let's let's play some makebelieve. you're a security security architect and you are tasked with designing a security system. We have our main network. This is our enterprise network, our production network, what have you. And inside this network, we have our systems and our tools that engineers use to do their daily jobs. We have our data. We have our company IP. And then we're tasked with isolating

this other network. And it's scary. It's so scary in fact that it doesn't feel good to simply stick it in another VLAN or another network separated by a firewall and call it good. We're concerned about pivot opportunities. We're concerned about zero day So, we start with an isolated air gap network. Sure, it might have internet access, but otherwise it's strictly isolated from your regular trusted network. And

we feel good, right? Not so much because then the requests start coming in. Okay, so we have the separate network. How are we supposed to manage network protocols? We need DNS. We need DHCP. We need NTP. Well, the best way we have today is to tie into our internal systems. We need network management. How are we pushing our our ales? How are we pushing our network configurations?

Well, can we just open up SSH into this system? Or can we open up our internal tools that handle all the config management, identity management? Are we going to have shared users across both systems? If so, then we need to wait to keep track of these physical security. How are we managing our badging systems? How are we managing our security cameras? What does that look like? This

the physical security team, they're not going to want to spin up an entirely new separate piece of architecture in in the isolated network data storage. The engineers working in the isolated network, they're going to want to use the internal data storage that they know and they love and that they're used to. Data transfer, business needs, vulnerability management. You may have a robust vulnerability management system, but where

does all of the resources for that system reside? In your regular If you're in security, you're probably internally laughing or crying because you recognize this pattern all too well. And if you're not in security, if you're a software engineer, if you're a hardware engineer, you've probably made one of these requests to your security And you've made those for legitimate reasons, right? You have work to do. You

have deadlines to meet and this is what you need to do that. Uh but what we have now is not an air gap network. We have a colander. I included this slide because it's just fun and I think it really drives home a point that true isolation is extraordinarily hard to achieve even at the physical layer. But honestly, if your air gap has been compromised by thermal

channels, you have a very different class of adversary. And I'll probably be seeing you talk at Defcon. For the rest of this talk, we are going to focus on the far more common and far more fixable problem, the operational holes that we poke in our own air gaps. So, we have our leaky cauldron. What do we do? Whether whether you're initially given the problem, okay, we need

to isolate and you start with the air gap or you inherit a network that you're told is isolated, you're told is secure, and you discover through auditing, etc. that it's it's not. The initial response might be great, let's cut off all access, let's call it good, we call it done. Uh but unfortunately we work in the real world. We work with engineers. We work with teams who

if we did this we'd have a massive massive impact on what they're able to do. Uh so we want to start by auditing may sound boring but this is a huge huge part of security. You want to map every crossing. You want to make sure you have nailed down what your firewall rules are, what your shared services are. if you have jump posts, what those jump posts

look like, etc. Uh, and then you want to classify, are the access points that are open, documented, and owned or have you just discovered them? Uh, and then think about the blast radius that you have. You can look at shared identity. Uh, you can say ask yourself, is this service gets compromised? What effect will it have on the broader company? Uh, and then you can look at

quick runs. Are there other firewall rules that haven't been touched in years? Uh, And then you can start thinking about uh some kind of controlled buffer zone. This is this is a core architectural idea we'll be building on. And instead of pretending that we have an air gap and we slowly erode it over exceptions and business use cases, we can explicitly design for controlled connectivity. We can

think of this as a DMZ specifically for internal isolation. DMZ typically is how you allow the outside world to interact with certain internal systems. And there are some key differences with here versus the DMZ. I think one of the largest differences is there's a lot more shared resources and shared um shared users that can go between the networks sometimes, but nothing passes between the isolated uh nothing

passes directly between the isolated segment and your internal systems. Instead, you have everything passed through this controlled buffer. Uh but there are some key considerations. I think the biggest consideration to think about is how you think about the trust of this controlled buffer zone. It's not as untrusted or high-risk as your isolated your your danger network, if you will. Um, but you also can't assume it's as

trusted as your internal regular normal network. Uh so we go back to our our leaky seieve. We have data blowing flowing both ways. But what if we started thinking about moving copies of our internal systems into our buffer zones? We could spin up for example new instances of authentication servers. High-risk users and devices can now have a dedicated set of O services to use. There is no

need to have our high-risk services be leveraging the same authentication services and the same backend as our internal services. And we now have a limited blast radius if this authentication server gets popped. The only access the hackers will have will be back into our high-risk zone. Another example here, license managers. Uh might seem like a fairly lowrisk item, but we continuously want to be cutting access off.

Okay, so this is great. We've we've solved the problem, right? Uh except I think we've just shifted the problem because eventually all of those services still are going to be managed and logged and monitored by Uh so the question morphs from how do we solve our leaky se to how do we manage connections between our trusted controlled buffer zone and our internal and there are several ways

again we go back to what is the actual risk of this intermediate zone and that's a question that that might look different depending on what the high-risk zone at the other end looks like. It might look different depending on what kind of data you're trying to protect, what what the actual reason of the high- risk is. Um, but there there are the the the um the key

thing to pull out of it is it is more trusted than your low risk or your high-risk zone. Uh and we can think about channeling everything that goes between our buff buffer zone and the regular network through authentication uh as well as authorization. Heavy heavy focus on monitoring uh and you can start exploring readonly Okay. So now we we have our mirrors, we have our caches, we

have our data storage. Uh but we still have some holes because sometimes an intermediary buffer still isn't feasible. So what are cases where we might still need direct access? Uh oftentimes the biggest the biggest problem the biggest challenge that typically comes up is that there are high service dependencies on the original network. Uh so the environment of the regular network is just too different from your controlled

buffer zone and the cost of bringing a new service into the controlled buffer zone is simply too high. Uh and suddenly you have if you if you fix the controlled buffer zone. You you have to open up maybe 10 new network ales back into your regular security zone. Uh other times you have financial constraints or just operational constraints. You may not have the people. It might not

be a priority. So there are actually a few options we can start exploring here as well. Uh we can if you have a unidirectional data pattern. So you only ever are trying to send data to your high-risk or you're only ever trying to send data back from your high-risk zone. Uh you can explore data dodes. They are a hardware enforced unidirectional data transfer. Uh or you can

start implementing a push from trust or pull from trust type of policy where you ensure traffic the network traffic itself is only ever originated and initiated from your standard network. Or you can also start exploring mediated access patterns. So a great example of these are proxies. Uh you can actually focus on terminating your connection from your high-risk zone into the controlled buffer. You can man in the

middle so you can inspect, filter and sanitize your traffic and then use that proxy to then initiate new connections to your regular network. And what have you done here? you've cut direct access uh which can prevent lateral movement. Uh if that's not feasible, you can also start exploring separate VIPs or separate instances within your regular system. So you can stand up again going back to license managers.

If we couldn't move a license manager into our controlled buffer zone, we could explore creating a separate instance of a license manager in our internal system and just make sure that we're pointing our our high risk traffic to that license manager. So again this is all about segmentation of services and reducing blast radius. Okay. Uh okay. So now we've once again reduced the direct access between these

two sites and we have known authenticated easy to monitor easy to audit paths between the two. So even if we're sending a lot of data, if we have a lot of connections and different types of data going through our proxies, going through our our buffered services, uh we have well-defined behavior that we expect to see. This makes it really easy to go to your detection and monitoring

team and explain to them, hey, can you can you build monitors for this? Can you build detections for this? uh instead of throwing the previous multitude of connections at them and asking them to define behavior based on that. Uh and then sort of the last piece of the puzzle is I talk a lot about the different trust levels and this is really possible when you start leveraging

separate certificate authorities. So ideally you would have different certificates be assigned to your different trust zones. This is getting into our zero trust networking. Uh this this leads to identity separation. Uh even if a certificate in your high risk zone gets compromised, makes it difficult for them to use that certificate in the rest of your network. Okay. So four main principles for sustainable isolation that I've talked

about. Assume compromise. these high-risk environments. Assume that every network device and every host in these networks has been compromised. Second one, minimize your transfer points. Make sure you're funneling all traffic through known, trusted, auditable proxies or buffers because then that can allow you better to monitor everything. And the last one is that you want to test and audit. do regular audits. Make sure make sure you know

what access you're allowing. But this can be hard, right? There's a lot of operational friction. You're constantly having to balance security with usability. You're going to come across a lot of legacy systems. The larger the company, the more of a wild west it can become. depending on the use case, you may have systems that you can't upgrade. Uh so you're just going to have to figure out

how to isolate them and how to progressively upgrade them. Uh and then exception creep, which I think is the biggest one. Every single request that's being made, you need to make sure you have proper justification. And if there's a certain point where your exceptions are exceeding the rule, that's a really solid indicator that you need to re-evaluate what your rules and your solutions are. Okay. So air

gaps, I think they're one of the most commonly known security uh strategies and they're actually in my opinion very terrible security strategies because they always always always erode through operational security. Uh an air gap you can't sustain is worse than no air gap at all because it gives you confidence you haven't earned. You want to design for the reality that information must flow and you want to

make every flow explicit, owned ex uh inspected and revocable. And while I use air gaps as an example, this line of thinking can really hold and should be applied to most approaches. Consider consider isolating networks to protect the world, not just the world uh protecting your assets against the uh and you can achieve this by taking an air gap light approach with an explicit way for traffic

to occur. Okay, but back to this slide. By now, we've made some significant progress in plugging our leaks. We have our replicated buffer hosted services. We have our proxies that terminate traffic and rebuild packets. But we still have some lingering holes in our calendar. And while it's not ultimately it's okay. Uh it's a largely unspoken aspect of the blue team world that you have to get used

to being uncomfortable with imperfection. A colleague of mine once told me said, "Aaron, I I could never do security. It's it's so frustrating. You're always you're never actually done. You're always solving for the next problem." and and there's you can't ever just land a project and walk away. There's always the next thing. Uh and personally, that's why I kind of love security. There's always you're always trying

to solve for the next problem. Uh and I've spent the past 30 minutes or so talking about some pretty high level solutions. But the truth is that driving remediation for each one of these data really could be a talk on its own. uh you can't solve everything everywhere all at once. But if you really lean into the empathy and collaboration across your different security teams, that's where

you can start the zero to one and begin landing the first of many iterative steps in your questions for secure by design. All right. So the title is talked uh the talk is titled porest by design and we got through some high level steps we can take to move towards secure by design. Again it's not always it's not always an easy path and there's a lot of

uh risk calculation that you have to do. Um, but this is kind of what a what a security architect goes through and depending on if it's an air gap network or just trying to figure out how to isolate different systems and services in a network. Um, this is the this is the kind of approach we take. Questions? Uh do you have any personal experience of any data

diode of vendor and do you recommend any of them because so far a lot of their product pages are not transparent about price or uh how available it is or how to even get them. They're all very, you know, contact us and we'll talk about it more later sort of thing. I I personally have not good question though. Um, out of curiosity, would you say that for

these uh air grip uh design, you also have a a bit louder, sorry about that. Uh, would you say that you have the documentation, you know, with incident management when they happen and and the whole protocols for recovery? Uh, do you apply the same principles for this or would you say they are slightly different? Um so the the question to make sure I understand correctly is uh

for these air gap networks do we apply the same incident management response that we do across other networks. >> Correct. I think it's a it's a bit of a yes and no. I think there's some high level concepts that you would still apply. Um but I think the it's a decision that every team every company has to make on their own. But sometimes there might be maybe

a higher uh or lower bar for a stronger response. Uh my question was uh the inspiration for creating this presentation was it derived from anything uh in your professional work and just curious >> um yeah sorry yeah I I would say it's pulled from many smaller many smaller interactions and many sort of ideas I've come across in my life in my professional life. Have you ever seen

um storage area networking used for data transfer between the domains? Um, this is something that I've used >> Uh, can you elaborate a little more on that question? >> So, like if you don't want to have if let's say you have a couple times a day where you need to transfer a large amount of data from your trusted network into your airgapped or um untrusted kind of

uh network. You can do that with storage area networking, right? So you can have a data volume that you plant your data onto in your trusted network and then have uh your uh your middle ground like server that's connected to that sand as well like over fiber or whatever. It can attach that that disk and then share that uh data o over the network in the uh

higher risk or airgapped environment. It's a great way to to to transfer data without about actually opening up network protocols. >> Ye yes. Um I think the the short answer is yes. The the longer answer is most often each data request is so unique uh that sometimes SANS doesn't work and sometimes you have to figure out a different issue. curious about your strategies potentially on getting the

operational team to sign off on basically doubling or tripling their workload [laughter] or at least uh like a success story. Um yeah. Yeah. Uh where to start? Um honestly I when I first started I was young and I was very idealistic and came out a lot of the problems with like it's my way, it's security way or the highway. Uh, and I got really good at saying

no. And while that did wonders, uh, then it got to a certain point where saying no was the wrong thing to do. Uh, and what I really what needed to learn how to do was say no, but you can't do it this way, but let's go let's go talk. Uh, and so I will say while that's not exactly applicable here, um, that attitude built a lot of

political capital for me and other teams outside of the security world learned to trust that I valued their work and I wasn't just there to stop their firewall rules. I wasn't just there to like block everything. I was there genuinely there to try and find creative ways to help them. Uh, and that's that's honestly probably my favorite part of security is working with really really talented engineers

and figuring out sometimes it takes going into the weeds of from end to end. how exactly is their data getting from this point to this point? Um, when it comes to asking for bigger lifts from the operational team, how can you do this? Uh, it takes work. It takes like it didn't happen overnight. Um, it takes figuring out how to talk to people. uh figuring really figuring

out the large picture and saying where do we where do we want to be in five years? Being able to communicate that and then then being able to take that step back and say okay we can't we can't have it be perfect right now. Probably in five years it's not going to be perfect but what can we do today? What are the three steps we can take

today to make that incremental step? And then it's getting really really used to and comfortable with taking those small steps and then communicating to the operations team the small steps are okay and the small steps are still a win for security. uh when you're talking about uh unidirectional data flows, do we have specific dedicated network protocols for that or are we just using UDP for unidirectional data

flows? >> Sorry. >> Yeah. Uh my question was about uh when you were talking about enforcing unidirectional data flows um from the untrusted network. Do we have specialized network protocols designed for that or are we just using something like UDP to accomplish that? Um I think when when we think about unidirectional it's less about network protocols like even with UDP you send the packet out but then

if you're expecting any kind of return uh suddenly you do have to open this weird firewall rule that basically says allow all UDP traffic. Um so it's less about uh I would say it's less about the network protocol and more about for example making sure that the TCP connections are originating from your trust environment. Making sure everything is originating from your trust environment. Uh >> thank you.

One more. >> Is this gonna make a big sound when I unplug? right. I'm I know the uh the closing ceremony is at 5, but we don't have to wait until 5 since she finished early. So, um I'm just going to talk briefly. Uh once again, thank you so much for uh joining us today. Uh it was an awesome two days. Uh we look forward to having

you. Um and what's next for us obviously hack the hack the base coming up on March 23rd. Um and then uh Pacific Hackers conference usually in the fall and uh we're trying to come up with the meetups in SoCal. So definitely uh sign up for our meetups. We'll be around and uh we'll try to be back here at at Sunset Con next year. So definitely follow us.

And as always, I want to say thank you to Scale for uh putting this together. This is an awesome conference. Uh so thank you. Um we're not coming we're not going to be here tomorrow. So if you wanted to buy a sweatshirt or a t-shirt, this is literally your you only have like an hour or so to do so. Okay. Um again, if you want to follow

us, please you have the uh the QR codes on the other table. And again, once again, thank you so much for coming. Uh, thank you for the support and we'll see you next year. All right. Thank you.