KEYNOTE: Alexandra Bell, The Security Implications of AI

In this talk, Alexandra Bell discusses the critical intersection of artificial intelligence with existential threats like nuclear proliferation, climate change, and biocontrol. She elaborates on the role of the Bulletin of Atomic Scientists, which serves to educate policymakers and the public on such risks and the importance of addressing them. The speaker provides insights into how AI can both mitigate and exacerbate these dangers, emphasizing the need for responsible programming and decision-making frameworks. She also highlights the complex implications of AI in nuclear command and control systems, the potential for misinformation, and the crucial need for public awareness and dialogue to guide ethical AI development. Bell stresses that collective agency in the current technological landscape is essential to shape a safer future.

So, a little bit about our speaker. As a noted policy expert and former diplomat, Alexandra Bell previously served as the deputy assistant secretary for nuclear affairs at the US Department of State. And today she is the president and CEO of the Bulletin of Atomic Scientists where she oversees the publishing programs management of the doomsday clock and a growing set of activities around nuclear risk, climate change, and

disruptive technologies. So please join me in giving a warm Drupalcon welcome to Alexandra Bell. Good morning everyone. It's great to be here at Drupal Con. I've heard it's the 25th birthday. Uh so happy birthday. I know some of you may be looking at your program and thinking, "Huh, what is she doing here?" Uh, I watched a video of a previous keynote speaker who was talking about the

connection between STEM and dance and I had two thoughts while watching. One, this is so inspiring. And two, I am literally a keeper of the doomsday clock. How am I not going to freak out this audience? Uh, I admit I am not a typical choice for a conference like this, but I also truly think uh that you all should know at least a little bit about the

things that I know. First, let me explain a little bit more about the bulletin. Founded in 1945 by Albert Einstein, Robert Oppenheimer, and University of Chicago scientists who helped develop the first nuclear weapons, The Bulletin is a nonprofit media outlet based here in Chicago down in Hyde Park that equips p the public policy makers and scientists with the information needed to reduce humanmade threats to existential uh

risks, nuclear, climate change, disruptive tech that includes biocurity and artificial intelligence. Now, Einstein and Oppenheimer could have just made an academic journal, uh, a closed circle of academics talking to themselves, but they purposefully chose to connect scientists with policy makers and the public to discuss and debate the scientific, military, diplomatic, environmental, and ethical implications of the new world they helped to create. Over the years, we've expanded

to cover other human cause existential risks, including climate change and AI. Uh, today we publish digitally in two formats on our open access website, which has more than 500,000 readers a month, uh, and in bimonthly magazine subscriptions, which can be found in 7,500 libraries around the world. We host the iconic doomsday clock set annually by our internationally recognized science and security board. And we provide regular online

and in-person events for the public, policy makers, and scientists to discuss existential risks and the possible paths for reducing those risks. Shameless plug, you can sign up for our newsletter and get all the latest on existential threats sent right to your inbox. Uh that may seem like a depressing thing to get on a Tuesday morning, but I assure you it is not. The bulletin doesn't aim to

scare people. Uh it happens of course. Uh but what we really seek to do is outline the nature of these risks we're facing and then the nature of the potential solutions. Most of all, we strive to ensure that everyone understands they have agency in these debates. So that's what I want to talk to you about today. You all have agency in how AI will affect our security

in the future. Of course, y'all are already seeing the benefit uh of AI in work and your daily lives. AI tools can uh give people the uh ability to leverage uh their own expertise over common tasks, increasing the amount of things that people can accomplish in any given day. AI is improving and maximizing the utility of tools we already have like Drupal. The most optimistic view is

that AI will improve everything. It will improve education. It will make everyone an expert, cure diseases, end poverty, roll back climate change, and even make us live longer. Sounds pretty great. On the flip side, did I do it? Nope. On the flip side, the pessimistic view is that AI will make us stupid and lazy, eliminate jobs, increase poverty, exacerbate disinformation, undermine democracy, and destroy any semblance of

personal private privacy that we have left. There is also the idea of self-aware machines taking over the planet, uh, Terminator in the Matrix. Honestly, if you went back in time uh to 1984 to tell James Cameron that his forthcoming action film, which was awesome, but also a little bit of a be movie, would one day be dominating diplomatic discussions in US UN bodies. I don't think he

would have believed you. Uh there is literally a movement called ban killer robots. But dangerous AI scenarios don't even need killer robots. Oxford professor Nick Bostonramm outlined a thought experiment called the paperclip maximizer. In it, the idea is that someone programs a super intelligent AI to maximize paperclip production. That AI has only that one instruction and it starts to do so with ruthless efficiency, drawing on and

diverting every possible available resource for making paper clips. Once humans try to stop this paperclip making fiasco that the AI decides that humans are a threat to its mission and that perhaps maybe humans could also be made into paper clips and thus humanity ends. Many people push back on this theory saying we would have controls and constraints along the way. Uh that may well be true but

it misses a key point. This extreme scenario shows how important it is to program with intent and clear purpose. In some ways, this uhi scenario is similar to the idea of being careful what you wish for or what you ask for. This mistake is actually pretty common in international security world, believe me. Uh overall, the increasing sophistication of large language models and their application and critical processes

coupled with lingering concerns about their accuracy and tendency to hallucinate have generated significant public debate over the potential risks of AI. The good news is that brilliant people have already been thinking about this for a long time and even more brilliant people are thinking about it now. The bad news is that not enough people are thinking about it particularly in governments and it the technology is actually

evolving faster than our ability to understand it much less control it. When it comes to the security implications of AI the truth is likely somewhere in between AI will make a perfect world and AI will kill us all. In terms of existential risks, the increasing inclusion of AI into our lives brings with it credible but uncertain risks. That's a reality that we're just going to have to

deal with. As someone who spends a lot of time thinking about the end of the world, I tend to come back to the advice of science fiction writer Douglas Adams. I will reference him a couple times today, but if you remember nothing else, remember, don't panic. Carry a towel. The towel in this case being a metaphor for preparedness and prepare we must. AI is going to affect

every human cause existential risk in both positive and negative ways. From nuclear risks to climate change to biocurity to disinformation to changes in our society as we know it. AI is and will be disruptive. In terms of nuclear risks, there are many issues to think through. First, how will AI affect proliferation? the spread of nuclear weapons. Can advanced computing power make nukes an easier process to build?

Uh, are we increasing the risk through the expansion of civil nuclear technology? After all, a civil nuclear capability can be used as a nuclear weapons starter kit. On the other hand, AI can potentially help us better track and and safeguard elicit nuclear materials, helping to prevent proliferation. In the same sense, AI might help us build better nuclear arms control agreements, allowing us to monitor items and activities

we could not in the past by enabling better verification systems. There's also the question of how AI is going to affect the weapons themselves. I had the privilege of leading the US delegation to discuss this issue with recognized nuclear weapons states. That's China, France, Russia, and the UK. All of us but Russia have now stated that we will maintain meaningful human control over nuclear launch authority. Happy

to talk about why Russia has said no in the Q&A. Uh that's a small step in a much longer and more complex road. As AI is more deeply integrated into nuclear command and control and communication often called NC3. Uh, we have to think about what potential aspects of AI nuclear integration will be the most unpredictable or potentially risky. Can AI improve decision-making in a nuclear crisis? Or

even with massive amounts of data, will it read a situation wrong? Over and over again, nuclear near misses have been averted by humans acting on instinct and actually breaking rules or protocols. In fact, I'll quickly tell you about a man who saved the world. And I'm guessing most of you have never heard of him. On the night of September 26th, 1983, a Soviet duty officer named Stannisloff

Petro was on watch monitoring a nuclear early warning system. The system detected up to five nuclear missile launches from the United States. The data was clear and it was on Petro to notify his superiors who would definitely move to launch a nuclear counterattack. The Soviet Union had just shot down a civilian Korean jetliner that flew over its territory, killing everyone on board. Tensions were high. But despite

this, Petro hesitated. He ignored protocol. He went with his gut. He reasoned that if the United States really wanted to start a nuclear war, we would send more than just five missiles. For context, at the time, we had 23,35 nukes. The Soviets had about 40,000. He also checked and saw that Soviet groundbased radar installations were not detecting any missiles at all. Taking a major risk, he decided

it was a false alarm and that he and it turns out he was right. Uh the Cosmos 1382 satellite, part of the Soviet early warning system, mistook the sun's reflections on the top of clouds for incoming missiles. The computer was wrong. The system was wrong. It's easy to see how mistakes like that might happen again as we increasingly transfer control over decisions to AI. There are some

who think we can program AI to detect false alarms. The problem is that to do so, AI will need a source of information that tells them that an attack is not really underway. If that source is compromised or spoofed or itself has an error, we are in some serious trouble. Petro was able to bring in external information into his decision-making process. An AI that is programmed to

accept information from a specific set of sensors cannot by design receive external information. This is not just an AI issue. We all tend to think that we can account for all possibilities, all variations to the norm. Douglas Adams had an idea about this too. Uh I bring it up often uh when dealing with nuclear issues when I served in the government. Uh, a common mistake that people

make, he said, when trying to design something completely foolproof is to underestimate the ingenuity of complete fools. Again, credible but uncertain risks are present with AI. And when it comes to nukes, the stakes could not be higher. Of course, AI will affect other major existential risks. When it comes to climate change, AI can be a tool for good, helping to optimize energy grids and green tech, improving

modeling and analysis, and advancing ideas for new resilience tools and techniques. Managing and mitigating the effects of climate change will take a global effort, and AI tools can and will help facilitate that effort. However, the demand for AI is a radically expanding demand for energy. Data centers are springing up throughout the world, drawing both power and water from communities. when these resources are already increasingly strained. Currently,

the demand must be met through the burning of fossil fuels or via nuclear power, which as I mentioned before carries with it proliferation and other safety risks. The Russian attacks in Ukraine on the Zaparisia power plant have brought to light just one risk. Further, the raw material required for AI related hardware uh has an environmentally damaging extraction process attached to it. Energy consuming manufacturing. It also contributes

to growing electronic waste problems. There are also indirect effects like increased cons uh consumption driven by improved more efficient production processes. When it comes to public health, AI can improve both preparedness and our response options by analyzing massive amounts of data and improving diagnostics. AI can help identify uh patterns to avert the next pandemic and improve the production of vaccines and medicines. AI can also be used

to strengthen our defense against biological weapons by bolstering our verification and monitoring capabilities, helping to catch or deter possible users of such weapons. However, AI can also be used to increase biological threats. AI assisted biological research can lead to intentional or accidental disasters. Possibly near the top of the terrifying meter is the potential for AI aided design of new pathogens to which humans have no effective defenses.

Given what the world collectively lived through just a handful of years ago, I probably don't need to convince you of the severe security implications of an AI enhanced pandemic. Still, if you don't feel like sleeping tonight, go check out the concept of mirror life. you will not on disinformation. Of course, we have already seen how disinformation complicated the management of COVID. We can expect more and worse

attacks on the truth in the future courtesy of AI. AI slop is now all around us, corroding people's trust in everything they see, read, or hear. When we cannot tell what is real, when we cannot agree on what is and is not true, how do we actually solve problems together? LLMs are actually breaking the connection between original fact-based analysis and content and its consumers. The bulletin has

already seen a loss in readers as LLM scrape our site for content and then sloppily repackage expert analysis into half-formed bullet points that miss key facts and data. It's true that the world's information ecosystem was already corrupted, but AI is accelerating the existing chaos and dysfunction, supercharging mis and disinformation campaigns, and undermining the fact-based public discussions required to address urgent threats like nuclear war, pandemics, and climate

change. And while labor trends admittedly are not an area of my expertise, I am very aware that large swasts of unemployed and potentially angry people idling around because they've lost their jobs due due to AI is a recipe for civil unrest. That kind of unrest can escalate quickly and since humanity has the capability to destroy itself, we need to be mindful about how technology is affecting our

society. There's no clear guide on how economic structures will evolve with the increased use of AI. But to be clear, we have managed and adapted to a myriad of techn technological revolutions that have rendered previous jobs and processes unnecessary. What we need to make sure of now is that we're taking the time to debate what is and isn't in our collective interests. move fast and break things

might sound super cool, but perhaps not when we're talking about norms, behaviors, and structures that govern human society. I've also heard a lot of talk about AI is just simply going to get rid of useless jobs. Here again, I'll reference Douglas Adams. He wrote about a planet that decided that it needed its top level thinkers and low-level practical workers, but it could live without what they deemed

as useless people. hairdressers, management consultants, marketing executives, the people who sanitized payoneses. Of course, this was written a little while ago, so there were still payoneses. Uh, those useless people got shipped off the planet and Adam mused that everyone who remained lived happily until everyone died of a plague that someone caught from an unsanitized pay phone. So perhaps here again, we should be careful about what we

wish for in terms of the future of our workforce and our economy. So, I've outlined a lot of threats and challenges that we're going to have to deal with, and I'm even managed to get multiple nerd references into the mix. What I want to make clear is that we will decide how AI affects our security. The choices we make, the leaders we elect, the businesses that we

support, that's all going to determine our future. We know that countries including the United States, Russia, and China seem to be running full tilt into an AI arms race that can affect our health, our environment, and our very lives. We know that businesses seem to be prioritizing speed, innovation, and profit over safety, and ethical and social considerations. We know that people are increasingly using LLMs and AI

agents without thinking about the potential negative consequences of those actions both in terms of theirelves and their communities. Managing these challenges will not be easy, but difficulty should never be an excuse for inaction. Going forward, countries will need to engage in a sustained and serious dialogue about standards, norms, behaviors, and regulations that can effectively manage the negative consequences of AI. Countries also need to realize that AI

isn't something that could be relegated to a subfolder of defense strategy. AI experts cannot be put in windows, windowless offices in the basement of a government building only to be trotted out when needed. And I'm not bitter at all because I'm a nuclear expert that was kept in a basement and trotted out when I was needed. Uh AI is changing the entire security environment. So successful strategies

will require breaking down silos or what the Pentagon likes to refer to as cylinders of excellence. We need to make sure I'm I'm glad I'm the first person to tell you that one. It's awesome, right? Uh we need to make sure that the right people are in the right rooms. In terms of specifics, countries, particularly the United States, Russia, and China, can engage in bilateral and multilateral

dialogues on meaningful guidelines on the incorporation of AI into their militaries, particularly into their nuclear command and control systems. The international community should also cooperate on meaningful measures to reduce the prospect that AI can be used to create bi biological threats. When it comes to climate, countries, industry, and community, they need to focus on how AI tools are being built, how they are being powered, and how

they are being used. The shift to renewables should be a priority, as should increased efforts to reduce the energy needs of AI tools themselves. To combat disinformation, governments should invest in rapid and secure communications tools. The US and and Russia figured this out actually a long time ago and have multiple duplicative and redundant secure communication systems. One of those lines is housed at the State Department. It's

called the NERK, the nuclear national and nuclear risk reduction center. Uh my the director of that office used to say uh we're connected to Moscow and I cannot guarantee you that the Russians aren't lying to you, but I can guarantee it is the Russians. You can see how that would be important. If weaponized disinformation is spreading online and in the media, a crisis is escalating, it's important

for government officials to know that they are actually reaching the right people. Uh, as an example, on the other side, a number of years ago, the Chinese conducted an anti-satellite test, creating tons of space debris, which is a whole another topic I could talk about. uh but uh some of their own space debris was about to hit one of their satellites. Uh we saw it uh better

observation capabilities. We wanted to warn them. It was a 12step process for us to get to the right person in the Chinese system to say, "Hey, move your satellite." Because if that tiny little bit of debris, I think it was only about two inches long, had hit that satellite, it probably would have exploded that satellite too, creating even more debris in space. So hotlines are important. Secure

communications absolutely necessary. We have to increase online literacy among the general public and in our schools. We need to create better detection tools for AI generated deep fakes and accountability for its purveyors. It will be important to support fact-based quality journalism like the bulletin and others. Uh otherwise uh and overall we need to rebuild trust in our media outlets and with our governments. Everyday people will need

to educate themselves and each other about the potential perils and pitfalls of using AI tools. Uh Stanford professor Herb Lynn, a very good person to to listen to on these issues, suggests prudent skepticism with regard to AI. And I think that's good advice to follow, particularly as this technology continues to evolve and improve. He explained that machine learning, the current paradigm for AI, is just large amounts

of data that enable computers to perform tasks without explicit instructions by generalizing or learning from patterns in that data with minimal human intervention. So at its core, machine learning is just still a way of programming computers. That means garbage in, garbage out still holds. He explains that machine learning can address unknown unknowns any better than people can. And as I noted before with Mr. Petro, machines can

be worse since they are inherently combine combined and chained to their own programming. He also notes that machine learning is as at its root just statistics. So perhaps it's helpful for people to think of machine learning as statistical, not actually smart. Demystifying AI and applying it judiciously is just one part of a larger process. The public and policymakers also need to hold industry's feet to the fire

when it comes to transparency and ethics. It should be made clear that security concerns should never take a backseat to politics or profit. It's 85 seconds to midnight. The not quite done yet. I got to scare you a little bit more. Uh the debate over why AI uh how and why AI is used is just the beginning. Despite the bulletin's best efforts to inform the public of

new dangers of the nuclear age, we have danced at the edge of the nuclear abyss far too many times. And unfortunately, we're heading back in that direction. Instead of learning from those past mistakes, we seem to be repeating them when it comes to how we're handling AI. The bulletin has been keeping the doomsday clock for the last 79 years, and I'm sorry to report that it is

closer to midnight than it has ever been. Uh, I've only touched on some of what has driven scientists and experts to this conclusion. The good news is that the clock does not predict the future. It is an assessment of our current reality. We have turned back the clock before and we can do it again. But not without people like you joining in the conversation and demanding action

from leaders at every level. To learn more about how to do that, here's the QR code to get connected to the bulletin. Uh and with that I want to thank you all for listening and I look forward to your questions. I believe there are microphones in the audience and uh I know some of you are itching to ask nuclear questions and that's totally okay. Hey, uh, I

was wondering if you could say a little bit more about the use of artificial intelligence in autonomous weapons such as drone systems, loitering munitions, etc. And also some of the wilder x-risk circles, there's been talk of like, what if you have drones that can make more drones? >> Um, is that a is that a threat that's on your radar or is that still science fiction? all the

dr I'll start with that one first. On the drones that can make more drones. Uh, I haven't heard of it, but I'm actually surprised at how often I run into new and scary things like mirror life that I didn't know about before. Uh, but I think uh overall drones and some of you I think work at the Pentagon. Uh, I can't tell you how many people uh

would would fuss that drone is an inaccurate term for what we're talking about unpiloted aerial systems. and I used to sit in those meetings and tell them the ship is sailed. They're called drones. Uh we haven't done nearly enough talking as an international community about the regulation of drones. We're actually seeing the application of drone warfare um massively expand in Ukraine, in Iran. The asymmetries that drone

technologies can provide to countries um needs to be acknowledged and studied. So I would say overall probably we are behind the ball in having governments talk to each other uh about the regulation uh of drone warfare and drone technology. Um some efforts have started but not nearly enough. So more to do on autonomous systems themselves. Uh and here's where the ban killer robots uh uh movement comes

in. There are people who are who are moving to ban any use of lethal autonomy. So, a weapon system that could decide when and where to use lethal lethal force without human assistance. Um, there are all sorts of moral, ethical, legal reasons to have that debate. My concern is that there are those who think there are legitimate and useful military purposes for the incorporation of autonomy into

systems. And I tend to think that when you go for a just ban it all approach as a technology is evolving, you'll tend to lose that argument and then there'll be no regulations in place. So what I think we really need to do uh in in terms of lethal autonomy is again start talking to each other uh start having Congress ask questions. You know, you used to

not be able to be on a relevant defense or appropriations committing in Congress and not be wellversed in all the security implications and issues of the day. That's no longer the case. So we actually have to be demanding that our leaders get up to speed. Ask industry, ask the military, uh ask the state department what is being done uh to open up and broaden a debate about

the future of lethal autonomy and and overall the implication that it the incorporation of AI into defense systems. So more work to be done uh and uh hopefully Terminator style killer robots are still a far far way off. I think over there was next. >> Hi. Um, you talked a little bit about deep fakes and accountability. So, I'm thinking about or wondering about what your stance is

on section 230 of the Communications Act that protects the service providers from what the users are posting on their platforms. >> Yep. Uh, not a lawyer. I I've been lawyer adjacent for many years. Uh, like I'm scientist adjacent. Uh but uh deep fakes I find extremely concerning in how quickly they could escalate a crisis. They're becoming increasingly uh sophisticated, increasingly hard to determine what is and is

not real. I could see a scenario and and it's not even a a deep fake scenario. There was a a chemical plant explosion uh in Lebanon a number of years ago. I don't know if you guys remember it, but any explosion of that size is going to cause a mushroom cloud. But because people saw a mushroom cloud, they started to freak out thinking that someone had used

a nuclear weapon. Now imagine someone does that purposefully shows multiple attacks um you know in an already intense environment. Does that trigger um you know potential action? You know, industry has to be a part of this conversation. And the idea that they can just say, "Oh, it's not my problem and free speech and we're just, you know, we're just the platform. Users have to, you know, patrol

themselves." That's that's not going to cut it anymore. When you can think about how quickly a deep fake can be used to start a war um to trigger a nuclear launch. uh there are more conversations to be had and more pressure that I think governments around the world have to be putting on industry um to actually come to the table uh and and potentially even have uh

consequences and accountability um to manage companies that refuse to police their sites >> in the I think you were next in the middle. >> So um I was in a session the other day and there was a demonstration of AI in Drupal and a prompt was entered and >> I'm sorry I'm having a little trouble hearing. I'm sorry. I was in a presentation uh yesterday and there

was a demonstration of where a prompt was entered into Drupal and uh it takes a moment for that prompt to complete and uh you know we could go into a discussion I would say it's primarily generating content or referencing content or calculating but the word that they used the little uh tag on the screen was thinking >> and I'd maintain that it's not thinking and so is

that just me being, you know, nitpicky or is there a benefit to drawing a distinction between what AI can and cannot do? >> Yeah. Uh you'll see a lot of uh experts in AI talk about this. A really good person uh to uh to read is Melanie Mitchell um who talks a lot. She's at the Santa Fe Institute. She's also on the science and security board of

the bulletin. talks a lot about the anthropomorphism uh that's happening around AI and and the sort of almost need and desire of people to to put these human um you know capabilities and ideas and and emotions onto machines. It's not surprising we do it to our dogs like we we do it all the time. uh but uh I think trying to to draw the distinction uh between

thinking and analyzing simply analyzing the data that's been put in as Herb Lynn said um you know this is this is confined to what goes in at this point of course there are lots of debates about artificial general intelligence I know there are a lot of people who think we are may already be there that sort of changes the situation uh but again uh you know that's

the need for the more sustained dialogue on the broad scope of implications of what we're doing. And as we push, push push to make AI bigger, better, faster, at what point are we going to get into a situation where we do have to start thinking about machines thinking, all right, you've been very patient in the middle there. So, um, as someone who thinks about the end of

the world every day, >> not every day, >> how do you continue to get up and do things? set said another way, what can individuals do to kind of alleviate their own existential dread and the existential dread of the people that are are around us and that we care about? >> Yeah. Uh, surprisingly not the first person to ask me that. Um, I think about the end

of the world not a as an inevitability but something that we're working to prevent. Um, and I'm fully convinced that because we created all of these existential threats, uh, human cause and sometimes they're called man-made and sorry guys, they're basically all man-made. uh existential risk. The uh the uh you know the issue is you know we we have the capabilities uh to deal with this. What we

lack uh is you know more knowledge uh and political will and uh you know so that's how I keep driving forward. Um, I I push ahead because I have to because we have to figure out how to solve these problems. And I I tend to put nuclear uh at the front. The other experts that I work with tend to see um you know, their particular expert area

as as the most pressing. Um but we've got to make progress on all of these things. Uh and I'm also heartened by the fact that we've we've fixed these things before. We had a ludicrous number of nuclear weapons in the world. Nearly 70,000. We're now at the simply extremely dangerous number of 13,000. Uh but when I look at the space again, we're about to go in the

wrong direction and that's where the public needs to come back in. Um step one is just simply being aware of the threats. A lot of people at the end of the Cold War thought, glad we're done with that nuke thing and just left the conversation. There's so many people that have been born since then that were never a part of the conversation in the first place. So,

I see my job with the bulletin. Pull people into the conversation. Give them the information they need. Tell them, "Unfortunately, I know there are a lot of things that are messing up your life right now, but you're going to have to think about this, too, because if we get this wrong, nothing else matters." Uh, so I just keep, you know, pushing a rock up the hill, which

may seem sophian, but if we stop pushing that rock, it's going to roll right back over us. So, I just keep pushing the rock. And I also give my time, like I give myself time. uh to uh you know take a day off, go see movies, go on a walk with my dog, have beers with my friends, you know, remember what it is that we're fighting for

in the first place. >> Yes. So, um you touched on the dangers of misinformation and disinformation as a byproduct of either nefarious actors or uh AIdriven hallucinations. I was wondering like one of the biggest issues is the fact that when people are using AI, they generally don't what they don't know. So, how do you, I guess, evaluate the veracity of the stuff that you kind of see

a uh from AI and kind of pick and choose like or determine what's true and what isn't? >> Yeah, I think that's where the prudent skepticism comes in uh of of and we need to incorporate this into our education processes. Countries like Finland have already done this. Online literacy, digital literacy is part of elementary school education. Uh I'm also happen to be half finishedish, so I'm super

biased about the awesome things that Finland does. But uh that's something that we need to do as a society to ask people to question uh things and and not just simply type a question in and assume it's the truth. Both AI uh companies leading AI LLM's uh anthropic open uh chat GPT have disclaimers that tell people, "Yeah, this thing messes up. You can't totally trust it. You

should be doing your own research." they they're fully upfront about it. Um maybe they need to be more upfront, but I think as a society, we have a personal responsibility to tell ourselves, our communities, our children, our families that they have to think about this as a tool, not a magic, you know, creature that can give us all that we want and tell us everything that we

need to know. Um it's simply a tool like a calculator, like a computer, like a word processor, like the printing press. uh and that it's up to us to actually distinguish what is and is not true. Think over there. >> Either one. >> Hi. Um you touched on a little bit about um setting standards for as AI continues to grow. Um but with any emerging technology where

companies and people, businesses where they throw uh capital, labor, resources into it to make the the best possible thing. There's also the point where things start to regress where we've reached a point and we've realized we could start cutting corners, less labor as we see with, you know, layoffs as is a consideration. But as there are decreasing standards uh of you know people getting hurt or even

for any of these things to just eat themselves and you know is the are these standards that you are considering are you considering this regression? Yeah, I think um uh another AI expert if you want to learn things uh Kate Crawford has talked uh about how we're sort of already at max data that that AI uh companies that LLM have sucked in everything there is to know

already and now are forced to create information. Uh and my first question to her was like for what purpose? And you know, she was talking about that there this drive that we need to be racing. We need to be first. We need to be better. If we don't do it, someone else will. We've got to push, push, push. And I think we haven't asked enough about what

are we pushing towards, you know, what what is the end goal here? Is it just to be first? Because just being first is how you get to things like nuclear arms races, uh, where you don't really think about the implications of what you're doing because you're just so set on an end goal that's sort of a murky place in the future. So I think you know governments,

communities need to be working directly with companies. Um there need to be more sustained conversations, more transparency and accountability. Uh but we know how to do things like this. The Chemical Weapons Convention probably aren't aware of it. Uh but bans the uh production, stockpiling, and use of chemical weapons. Um, governments actually work directly with the chemical industry uh to create that treaty because uh intrusive inspections um

the ability uh you know to to to report um all of that had to be worked out with industry. Government saw if they were sort of just imposing these guidelines, it would be hard to enforce. They needed the buyin of this industry to do it well. And that treaty is actually even with its faults the vanguard of arms control agreements and then the sort of level of

verification and monitoring that we aspire to. Um so we can take lessons learned from other industries where we sense dangers uh and wanted to control them but we actually did it as a as a a group project uh which takes time and patience and and those two things are in short supply right now. Uh but again, you know, we don't want to make the same mistakes with

AI that we made with some of these other dangerous technologies and we have the ability to actually get in front of some problems. So, we just need to demand that. >> Hello. Thank you very much for coming and talking to us. We are here as a community of people building free software and one of the topics I've noticed is that we have been prioritizing building AI tools

as a community and I'd like to think that that means something that somehow us being together building things is important but I wonder if that's a naive way to think and if you think that our actions, you know, either could make things better or worse as we create software that allows people to create websites. >> Yeah. No, thanks for that. And I I like that that you

brought up the idea of a community. I was actually telling Avi when I walked in, there was like a like a actually very positive vibe as I was walking in today and I saw everybody just sort of talking and um interacting and and I you know, Abby talked about it like a community. Um, it's weird. The nuke community, uh, the people focused on nuclear weapons also refer

to themselves as such. Uh, which seems sort of weird for such a dark thing, but we feel like we're all in it together. Uh, and not all of us agree on the same sets of tools we should be using to reduce nuclear threats. But we all agree that prevention of nuclear use, prevention of nuclear war is what we're trying to do together as a community. And I

think that that sort of uh the common cause there actually makes for better discourse, better understanding of solutions. And so if I look out at this community, I think it is that interconnection between each other where you can talk when where you can think about hey is this the wrong thing that the push against group think. I I think one of the problems with a lot of

the security discussions that we're having it's an increasingly insular it was always an insular group of people but it's increasingly so and it sort of pushes against uh you know good innovative solutions. So, I would say, you know, you're on the right track as long as you're aware of the implications um of how AI can affect all of these other things, you know, then you're doing the

right thing. >> Hello. Um you spoke a lot about uh AI threats on major infrastructure like nuclear power plants and whatnot. Um, how can we deal with that at the the smaller scale like where we see, you know, a few a kid in his basement with a clawed coat or something like that uh holding Fortune 500 companies hostage >> and amplifying what would be, you know, typical

bad actors uh agenda at the small and the medium scale up to the the larger scale. How can we as developers kind of model that interaction and like how can we steal ourselves against these kind of smaller scale to medium scale attacks and then how can this eventually tie into larger policy makers like uh like nuclear security and all that and is there modeling that we can

see both ways? >> Yeah. Uh I mean I think you would know better than I as far as like the the the detection, the forensics, the monitoring that could go in uh you know sort of at the low levels. I think uh law enforcement mechanisms um probably do not have or law enforcement um institutions around the world probably do not have enough experts um to credibly do

this right now. Um so the incorporation that's why I was talking earlier about having the right people in the right rooms that extends all the way down to how we're tracking um potential uh you know individual use of AI weapons I guess is is what you're uh referring to. And overall we have to think about how AI is going to create asymmetries in in possible um conflicts

in in warfare. Uh the ability of somebody in a windowless basement uh you know actually attacking you know a government. We've seen um commercial attacks. There was the uh incident where it looked like the North Koreans actually attacked Sony uh pictures in general. They were super mad about a movie. Um but you can see how these asymmetries uh governments aren't necessarily prepared for because they think about

conflict in terms of kinetic back and forth. We're we're not even sure of some of the lowlevel cyber attacks we've seen, you know, between countries actually. Does that constitute an attack? Does that trigger things like our alliance capabilities? I think about all the time too as as countries like the United States, China, Russia, others continue to invest um in AI uh in you know enhanced weaponry. Are

we actually turning nukes into sort of a poor man's weapon? Um this is 1940s technology. They made nukes without a calculator. I've seen where they put it together in New Mexico. It's just a just a house with like a bunch of wooden tables. Um they uh you know this is not exactly out of the realm. uh the bombs themselves, the file material is harder to make. AI

could change that. But um you know, are we pushing so far towards uh you know, using these capabilities for warfare that we actually encourage other countries to go with the very effectively blunt instrument that is nuclear weapons. So all of these things have to be discussed across the scale. you know, we have little things all the time like having an AI expert in a law enforcement organization

that can help with things like forensics. Um, when I was in the ad a Biden administration and we were trying to get the Chinese into a arms control discussion, it occurred to me that we had actually spent years training Russian language translators to actually understand how the Russians talk in their native language about nuclear weapons. we have a single person down in the interpreter's room that knew

Mandarin and also how the Chinese talk about nuclear weapons. So, it's just it's up to us to make sure that we're ensuring uh that we're getting in front of some of these problems and some of it is just basic brass tax like do you have AI experts on hand? >> Thank you. >> Hey, I love the Douglas Adams references. Those are great. Uh this is a nuclear

weapons question. Uh, what do you think about the movie on Netflix called House of Dynamite? Was that an accurate representation? The the Pentagon tried to play that down and say it was really not a big deal or it was exaggerated. What did you What do you and the people there at your newsletter think of that? so I'm a a movie buff, so I'm always excited when uh

movies come out on issues that I that I care about. And on nuclear weapons, it had been a bit when Oppenheimer came out. Um, it had been about 20 years since there was a major Hollywood motion picture where nuclear weapons were sort of at the center. And even with Oenheimer, it was more about him than than about nuclear weapons. Uh, the last movie uh that I could

recall was 2003 uh some of all fears, which is a pretty good movie. I don't think Ben Affleck is a really great Jack Ryan, but it's a pretty good movie. You should watch it. Um but uh House of Dynamite comes along uh you know shortly on the heels of Oppenheimer not about um you know a particular person that was a part of creating the nuclear age um

but rather you know the threats that are around us all the time. Um I got to screen the movie at the MPA uh in Washington I think in August. The movie came out in October and it was just I think four of us in the room uh and a security guard had from the MPA had come in to watch it with us and you know opening scenes

my first thought was oh my god are they actually at Fort Gley? I've been up to Fort Gley. Like they did a great job of making it look like Fort Gley where our missile defense interceptors are. Uh when they went down into uh sort of the bunker at Stratcom I was like wow they've either been in this room or someone told them exactly what this looks like.

Um, so they put a lot of effort into the realism uh of it, the process of, you know, the security line to get into the wizard, the White House situation room. Um, you know, they put a high premium on accuracy. I've talked to Katherryn Bigalow. I've talked to Noah Oppenheim, the writer, and they were very specific about the reason they didn't identify where the threat came from

is they didn't want to make a movie about us versus them. And they wanted to make a movie about the process by which a nuclear situation could escalate. How quickly a president would have to make a decision. How many layers upon layers uh of government agencies would be involved uh in a process where we detected an incoming nuke. Um there are lots of people who quibble with

with the details. We've actually there are a ton of articles in the bulletin of people kind of arguing uh back and forth. But for me, the important part um was that millions of people watched that movie and for the first time were confronted with the reality of a nuclear weapon. Uh you know, Russia's got 1550 of them just pointed at us right now. Not pointed at us,

but ready to shoot at us um you know, at a at a moment's notice. And we've lived with what President Kennedy called uh uh the sword of Damocles hanging over our head, ready to be cut by the slanderous of threads uh by ready to be cut by madness, miscalculation um or mistake. And uh that's the kind of of thing that I'm glad that the director and writer

brought back to the forefront. I'm hoping, you know, more people uh will watch some of the classics. If you haven't seen Doctor Strange Love, I I highly recommend it. But, you know, I'm aware that that the connection between arts and science is one that could be useful. The Doomsday Clock itself is is an iteration of that, a a symbol that can help people uh relate to the

existential threats that we see. Um, so overall, I'm pro, but uh definitely uh if you want to see the discourse on that, just uh jump into the bulletin. I also think the critics that I've heard um say well it can happen like that are also the critics there's a book called nuclear war a scenario uh similar in scope and I heard people say well it wouldn't happen

like that and the thing I kept coming back to is but something could happen uh something has happened many times over and we've escaped every single time but eventually luck does run out it's a statistical you know truism luck will eventually run out. The chances of a nuclear attack is not zero, which is why we need to be working harder, more every day. And it starts with

the public being more aware so movies can help. Every time there's been a push back on nuclear risks over the course of the last 80 years, it's been because the public has been demanding that their leaders take >> Thank you. Um, I just wanted to say thank you for everything that you do. Thank you for being here. Y'all are rock stars in certain circles. Um, so my

question is, uh, I guess In risk management in general, a big part of mitigating a threat is understanding it, understanding how it works, understanding what the likely outcomes are going to be. Um, so as researchers is on us to use these tools to figure out what might happen in ways that people could either accidentally misuse them or maliciously misuse them. But then the the thing that I

always go through is that it's learning back like as researchers the the queries that we use are going back into those data sets. Do you have any thoughts or recommendations on how to handle that problem space >> to to sort of identify the problem and then work back? Yeah, I mean I I think that's the key is sort of being fully aware uh of of the threat

space that's there, identifying, you know, steps that could be taken. And I I think you know the the military governments around the world they war game all the time. Uh I almost think sometimes can we like play war games but like a prevention of war game like like we actually like model out how we you know we we always start these scenarios mid chaos. And I would

like to do more exploration when it comes to disruptive tech, when it comes to news, when it comes to climate change um you know escalating situations as to like what are the things that we can do and it just starts with basics. the the conversation to get the US, France, China, and the UK to say that they would have meaningful human control over nuclear launch authority came

from simply talking to each other about who it was in their country that has the ability to launch a nuclear weapon. Uh, and everyone at the table said it is their head of their head of government. It is not their head of state. The Brits were very quick to say their king or queen is not allowed to launch the nukes. So, head of government, who makes that

decision? And as we were sitting there in a relatively tense room, sort of occurred to me, we've already then decided that AI will not decide that for us. Uh for better or for worse, you could say it's not it's cold comfort that a human, not a machine, would choose to start a nuclear war. But uh nevertheless, we we had already determined something about the AI problem that

had nothing to do with the definition of AI, nothing to do uh with how we manage AI in the future, but something that we can actually like get our hands around. And uh and that was the start of what I hope is a longer conversation uh by sort of identifying the risks and then figuring out step at a time what we can do. I might have not

answered your question. >> I I wasn't thinking so much at the state level as in so uh infosc is a part of my role. So I was thinking um in terms of the threats to existing websites for example, the threats to companies with their private data and so forth. >> Um the the fact that >> if you say war game to use your your phrasing war game

something out that information has gone back into the LLM then and it it then knows that scenario. >> so yeah I mean I think the same thing applies like government level and and sometimes people think like negotiating at a table is uh you know some sort of high level thing you know like you're still just sitting at the table with people who are representing their countries but

you're still trying to work with an individual. So I think you know as far as communities that are working through these threats the sharing of information uh that the sharing of of lessons learned of problems identified um has to be a part of the process that that that the you know AI has changed our entire security environment but also just it's changing everything. Um so I think

it's incumbent upon all levels of people working with this tool um to communicate with each other uh to talk with each other and when necessary to involve um you know government officials at every level uh on how we can sort of build structures, norms, behaviors that work for everyone >> Hello. uh you already touched on this a little bit I think but um I was curious about

if there's any ongoing discussion about um inherent bias because you already said that all the information has already been ingested quote unquote but how does um the potential bias of the information affect ongoing discussions and cultural shifts and then I leave you with black mirror sci-fi it reference Black Mirror. No, certainly certainly a good one. Um, you know, and and I think in in some sense to

go back to to the uh to the film question, you know, I've seen, you know, pop culture, media, you know, various forms, TV, books take on these issues in a in a more serious fashion than the governments uh who need to be worrying about these issues have. Um the questions that are being asked uh in some of these shows are the questions that we need to be

asking in real life every day. Uh in terms of bias, this concerns me all the time. Um you know, of course there's the the sort of programming level, but if we're relying on tools that are programmed by people that inherently misunderstand or or have biases against how they're interpreting the world, um that's a problem. we we we don't even fully understand our adversaries from a defense perspective.

Um, one of the problems that I I saw with the Chinese and was trying to work through with my Chinese counterparts uh was them saying they couldn't do things with us um because they don't trust us. And I was saying, well, we can't trust each other until we do things. It's the doing things that create the trust. And so, we had a fundamental like different starting point

for how you get to that level. Now, if you have somebody who's who who didn't bother to have that conversation uh with the Chinese, they would look at the situation and say the Chinese aren't interested in solving this problem as opposed to they don't agree with where we start. And those are two very different things and the first determination that they don't want to do anything is

how you can end up in a situation where crisis problems are escalating between two countries because they haven't bothered to understand each other. So, I would see that applying in tech world as well. uh that what we're bringing into this and as Herb Lynn says it's garbage in garbage out. If if you if you don't have a full picture and understanding, you're not going to to program

properly. >> Um so, uh I've been a longtime subscriber, so first off, uh sincerely thank you for all the anxiety. Um it's been much appreciated. Um but, uh you know, looking at the clock over the past, you know, many years, I mean, we're at the point now we're making small minute jumps, one minute, two minute at a time. um it kind of feels like you're writing a

poster in really big letters and now you're running out of space and you have to make them a lot smaller. Um have you thought about you know making a one-time reset of the clock um to kind of give people who aren't so familiar with the clock um you know see those larger jumps again um to help feel the criticality a little bit more um because if they're

not very familiar with it, you know, one second here, two second there um might not feel like a big change when realistically it's huge. >> Yep. Uh, no, thank you for that. I I I get a lot of clock questions. Uh, you might not be surprised. I've had people say, "We should be using a sund dial." I still don't really understand how that works. Uh, other people

are like, "You should be using a basian statistical analysis." And we're like, "We're not actually trying to to to to predict the end of the world. We're actually trying to just tell people things are bad and we should be moving in the different direction." Um I also have heard people say uh you know we use an analog clock and that kids today want to understand that. I

was like yeah we don't build pyramids either but people understand what pyramids are. I I you know the the clock the clock works but uh I think um what we're trying to do is simply relay the level of threat. And believe me, you don't want to be in a room for a bunch of astrophysicists um and theoretical physicists debating the difference of of a unit of measure

of second if you've got two minutes left on the clock or 20 minutes left on the clock because that could take up a whole afternoon, believe me. Um but what we're trying to show with the clock and where it is set right now, um and the idea that things are worse than they have ever been, um it's it's the trajectory that we're on and continue to go

on. Um, and what we've seen over and over again, these threats are increasing. We're fully aware of the threats. They're not taking us by surprise. Uh, but what we lack is is is political will to actually do things about it. And, you know, sometimes people ask like, well, wasn't it worse in the Cuban missile crisis? And I, you know, point out that the clock is about long-term

trends. The Cuban Missile Crisis started and ended in about 13 days, PS, between two issues of the bulletin being being published. And at the time a lot of us didn't know how bad it was and how close we came to disaster. But on the immediate heels of that, President Kennedy, President Cruchef, British officials immediately got to the table and with a year had negotiated uh the first

major arms control agreement which was the limited nuclear testban treaty. Um we had people who saw the threat and were moving to fix it. And to a certain extent, the 85 on the clock right now is is a representation of all of these threats getting worse, interconnecting with each other, layer on top disinformation and a move across the world to a zero sum nationalistic approach. You know,

you win, I win, you lose kind of uh governance structure. You know, that's what the clock is trying to, you know, represent. and and hopefully when people look at it uh you know they they think about that first and you know welcome suggestions about how we could do it better but right now like it gets to millions of people every year and I hope some of those

million people uh millions of people actually start reading the bulletin actually start asking questions um and that's what we're really focused on and uh you know I'll keep thinking about the sundial >> thank you >> I think just the last one yeah you got >> oh thank you for uh being here. Uh but uh earlier you spoke about how um AI was summarizing some information from the

bulletin but missing key bits of information. So with these versions, >> sorry, I'm having a little trouble hearing you. >> All right. Earlier you spoke about how AI was summarizing information from the bulletin but missing key points of information. So with these summarized watered down versions uh that AI scrapes, I know it can lead to model collapse, but how does that impact the risk of misinformation and

what regulation should we be headed towards? >> Yep. I um so yeah, it's I don't even think it's, you know, intentional by the LLM to to take an article, you know, summarize it down uh to something more accessible. And and to me, the key thing is people getting this information. Uh it would be nice if people scraping us would be paying us for it. Uh but there's

a larger New York Times lawsuit uh that we're watching on on that account. Uh but but it's more that you're getting maybe the wrong pieces. Uh and and to be clear, I think AI can be a tool for us. We're actually about to launch um an an AI bot that can read you our articles. If you can't be bothered to to read them, we can have them

read to you. I really wanted to go with the British accent because I think all of this sounds better with a British accent. But uh we just went with a standard Mid-Atlantic. Uh but even like listening to the bot sometimes it emphasized weird words. It it you know sort of didn't talk at a cadence that makes sense to people. And so I think they're still learning and

adapting uh that goes and and if it's if it's you know summarizing in a way that's useful that's fine but you're still not getting the full picture. Uh, and what I'm worried about is that people will just take bits and pieces of information that LLMs give them and not ever actually learn the full context and scope uh, of what uh, you know, again going back to the

the Chinese example, uh, you hear a lot, China doesn't want to do arms control with us, and there could be an article that's talking about that and and spits out that as as a summary, but that doesn't get to my point about the fact that we've got, you know, sort of a a bridge to gap between us. um on how we actually work problems collectively. So maybe

it's not that China doesn't want to do it, we just haven't figured out a way to do it with them yet in a way that they find value. Uh and it's that nuance that I'm worried about missing uh with some of these. So, but I, you know, I'm mindful too that it's our responsibility uh as a publication to actually meet people where they are uh and to

actually create content like having bots read things to you, but also creating more video content uh more programs where people can actually watch and interact with experts uh and maybe that's the way uh to to adapt uh to this growing world. LLM, people are increasingly relying on LLMs. Uh, so I don't think it's like the end of publications, but I think there are a lot of questions

that we're going to have to work through both with the companies with, you know, making these models, scraping these sites, but also how we get information to people in general, which is why you should all sign up for the bulletin, uh, and just get it directly to your inbox so you can learn about the end of the world as often as I do. I really appreciate everybody

taking the time to listen, uh, and good luck with the rest of the conference.