AI, elections and democracy: Rebuilding trust in the age of algorithms

This talk discusses the critical relationship between artificial intelligence (AI) and democracy, emphasizing the impact of AI on democratic trust and participation. The speaker highlights cases where biased algorithms have harmed vulnerable communities, such as the child benefit fraud scandal in the Netherlands, leading to significant social and political consequences. They explore how AI shapes the information environment, amplifying polarization and confusion among voters. Furthermore, the speaker addresses the challenges of using AI in public administration, including issues with facial recognition technology and proprietary algorithms. They propose several principles for a democratic AI, such as explainability, human oversight, and interoperability, asserting the necessity of transparent and accountable systems to sustain public trust in democratic institutions.

[music] >> Imagine you apply for benefit that you're entitled to. This could be housing, welfare, unemployment, or even a visa, and you're being refused. And you ask why, and you're being told, "Well, the system made the decision." You're You ask if you can understand how the decision was made, but you're being told, "No, that's proprietary." You ask how you can appeal, and you're being told to fill

in a form, and the form goes back to the same system. And unfortunately, this is not a hypothetical. This has actually happened in the Netherlands with uh child benefit fraud, where thousands of families were wrongfully accused of committing uh um by an algorithm um that determined that, and those families were mostly from minority backgrounds in um the Netherlands and held dual citizenship. Um and that not only

meant for them that they had to pay back thousands of euros to the government, they also lost homes because of these financial burdens, and families were taken into care, and essentially, that also brought down the government in the Netherlands. And the system, or this algorithm that was used, was not only biased, um it was opaque, and it was also unaccountable. And that did not only cause harms

to the many families that it destroyed, it also caused harm to our democracies by breaking the trust in the institutions that are meant to serve us. And this is what I want to talk to you about today, not whether AI is good or bad for our democracy, but what democracy needs from AI and how we can rebuild that trust. because democracy runs on trust. And AI now

sits at points where citizens meet the state, uh which means it sits at points where trust is either built or lost. So, I'm a professional from the democracy field. I'm not a technical expert. I work at International IDEA, as Michael already mentioned, which is an intergovernmental organization with a mandate to strengthen and promote democracies all around the world. And we work with governments, electoral bodies, and civil

societies on how technologies impact our democracies. So, again, I'm not a technical expert, but I'm here to tell you now what um I think democracy needs from AI. Sorry. let me start by outlining a few areas on how AI already shapes democracies today. There are plenty of areas, but because I also want to spend more time on how we can solve these tensions between AI and uh

let me only mention three big ones that I think are really important. And the first one is our information environment. Because AI puts our information environment and also our integrity under significant pressure. And this is not what many may think because um AI persuades voters or makes people believe into something else they have not um believed before, but we at IDEA think that the problem is more

um polarization, public polarization, and voter confusion. And here, for example, we can also look at Hungary's recent election, where uh the political campaigns online have been full of AI-generated and manipulated content, but first analysis um questioned whether that actually changed to or whether they actually had an influence on people believing in something that they have not believed before. And whether instead it uh perhaps did more um

reinforce existing fears and um hardened political camps. Um and this is not to dismiss the effect of um disinformation, um but it just means that methodologically we perhaps cannot um measure the impact of disinformation campaigns through AI-generated content on the outcomes of an election, um and we have to look at um other things. But certainly we can see that AI is widening the Overton window of what

is admissible, and also that one narrative can appear at different forms at once. Um it can be an audio message, what we see a lot on social media, or video clip, or also an answer of a chatbot. And so with AI, um it's not only changing the speed and scale in which content is produced, um including disinformation, but it's also flipping the economics of our information environment,

because attackers have to spend now less time producing content that they can also use for, for example, disinformation campaigns. Um while on the response side, institutions, the media, civil society organizations have to spend much more of their time on um verifying and or tracing in the first place and then verifying information. Um and it makes it much harder for them to respond before public trust is damaged.

And on top of that, the problem is not only for democracies that there is now false information, but also big that genuine and authentic uh information becomes easier to deny and um is being doubted by the public. But we should also look beyond the creation of side of AI because generative AI only uh produces or touches upon the um content side. Yeah, now we're in the right

side. Um platform design and algorithms that tend to favor engagement over democratic health uh determine whether a piece of content stays marginal or it reaches millions. And then there are many actors who exploit AI to attack the vulnerabilities in our democracies and who want to pollute the information environment, but there's also many actors that just do it for the money because attention pays, clicks pay. And there

while the motives may be commercial, the damage is still democratic. And yeah, all three layers are also largely opaque to the public. And the public has very little ability to scrutinize these systems. And then I want to speak about the use of AI in public administration. We also written extensively about this in a report that you can find on our website which is called rights in the

digital age, but I've also linked this later in the slides. Um, but I want to mention only a few of them today. So, one is facial recognition technology. Unfortunately, we still see that these technologies tend to be very prone to error and bias. For example, they work quite well with subjects that of of lighter skin, but with subjects of darker skin, there's much more or the error

rate is much higher. And they also work better on male subjects than on female subjects. And this is very problematic when it has impact on our rights and our life. Because in the US for example, three black men were wrongfully accused and arrested because of an error in the facial recognition system. And that's why when for example AI is also being used for migration and asylum decisions,

that's also a very high risk area because again, it's these are decisions about our human life and about our fundamental human rights. And then I also want to speak about the compass system which is a system used in predictive sentencing. It's a system an um based on different um facts determines whether a defender is likely or the likelihood to re-offend. And based on that score, the courts,

the judges then determine the length of um a prison sentence. And this is also something that was challenged by um defendant in the US. Um but he was told that um he cannot or this defendant wanted to understand how the algorithm works. And he was told this is proprietary and not even the courts are allowed to see the methodology behind the system. Um which again is I

think very um difficult from a democratic point of view that there is so much uh priority given over um trade secrets um when it there's like fundamental rights at stakes. and here I also want to mention uh the importance or well um no, sorry. I jumped um but uh in Netherlands, for example, they solved this by um or they tried to solve this after scandal by implementing

transparent algorithm transparency registers. Uh which also become now, I mean, they were first in a piloting phase, but this will now also become mandatory for high-risk systems in line with EU uh with the EU AI Act. Um and while this is a good ambition, it's questionable how much this actually helps because these registers only tell you that a public administration has used um an algorithm to make

a decision. And at most, it explains in very broad terms how the system works. But it does not really give any insights on the code that um or the how the code is written doesn't give insights on the training data and how the internal logic to truly make the system auditable. So, this transparency register uh would have probably also not really helped in the defendant in the

Compass case. Um and yeah, I want to mention here the importance of transparency not only uh for the code, but also for the training data because yeah, a code can be written flawlessly, but errors can still occur in the training data. Um and here we can distinguish between sort of two types of transparency. One is vendor transparency, meaning that the public body that uses the tool has

enough information on the design, development, and performance of the system in order to adequately oversee and procure it. we also need public transparency um to a degree and I will talk more about this also later um where there's enough information that's also being communicated to the public to hold the um their governments accountable and to adequately scrutinize the processes and decisions. Um and then thirdly, I want

to talk about political participation and exclusion because AI does not affect all political actors equally. And here I think the numbers are really shocking because 96% of all deep um deepfakes online depict women in non-consensual sexual imagery. And this is also something that we at IDEA have looked at in the Brazil or Brazil's 2024 municipality election where we saw that this was a very common tactic to

drive female political candidates out of um public spaces. Um and this does not um or this already starts at a very young age um unfortunately. A study a global study of uh 14,000 uh girls um underage girls um across 22 countries found that half of them have been harassed by for posting online and that discour- or severely discouraged 20% of them to ever um post publicly or

post their political opinions online. so this is very problematic from a democratic point of view because if 20 or if 50% half of population doesn't feel comfortable uh participating in online spaces which are now a huge democratic platform um that's very worrisome. but I also I mean I've talked now a lot about the challenges of AI to democracy but I also don't think or want to um

give the impression that technology is all doom and gloom because technology has also always been part of how democracies can improve themselves. And um we do see a lot of benefits in in technology for a democracy. To mention only a few of them, for example, we have in we have benefits in political finance. Uh the UK is uh currently the is currently exploring to, um, have AI

tools help them with the administrative processes of, uh, auditing campaign finance. And, um, then we also have benefits, um, in when it comes to more meaningful public, um, participation in policy making. For example, Taiwan has, uh, used an AI tool to help them cluster and summarize citizen input. And then we also have, uh, benefit in, in electoral administration, which can be very burdensome processes. And here AI

tools have been used to, clean up voter lists where sometimes there have been duplicate entries or there have been eligible voters, uh, but have not been registered yet. Um, so I'm really all for, um, AI in democratic life, but I think it's we need to have the necessary safeguards and principles that make those gains that I mentioned here and those benefits possible. Because it's just like with

a seatbelt, like the car was never the problem, but the problem was using a very powerful technology without enough protection. And, uh, the seatbelt also did not slow down a car, um, it only made driving safer. Safer, and I think AI is exactly also at this moment right now. So, uh, what does a democracy need from AI? and here I want to mention eight principles for a

democratic AI, which are explainability, human oversight, making sure that always there's a human staying in the loop, data protection and privacy, contestability, and accountability. And I want to spend a little bit more time talking about uh, transparency, audibility, and interoperability. And transparency is a really important principle that's also mentioned in a lot of international human rights frameworks and also embraced by international bodies such as, for example,

the UNESCO recommendations of the ethics of AI or the Council of Europe framework convention of AI or also the Open Government Partnership. I already said a bit before about transparency, but let me add two more thoughts here. Because I think there's also a fairness issue here. In some instances and with some public bodies that we've worked with, vendors train their their system on public data, but then

they make it proprietary and sell the insights back to the state. And that's why we say procurement rules need to from the start already protect public rights over trade secret and find solutions that um that reflect that. I understand that in some instances um it's important also to protect intellectual property or also, for example, in contexts where um the data is very sensitive. Um when it's about

personal data, then of course full transparency is difficult. But the answer's not full secrecy, um but instead we should look at qualified transparency, which means still documenting everything clearly. And another solution could, for example, be to give independent audit auditors access under confidentiality. Um because if we think of audibility, this is something that we have in so many other areas that affect human life and um our

rights at scale, for example, in pharmaceuticals or also in food safety. So, why don't we do this also for AI? And then last, interoperability. This may be less unknown as a democratic principle, but if democratic institutions depend on systems that cannot work together, also with already pre-existing systems that they have in place, or lock them into one vendor, then they lose control. And you cannot really have

real oversight of a system if you can only audit a system with the vendor's own tools. Uh so, interoperability is very important here because it helps institutions stay independent. It lets them switch. It fixes problems, and that ability to also always correct course is a core part of our democracy. So, I want to end with two messages, and I hope you can take them away for your

work. Uh first is on time because democratic institutions operate on time scales of decades and centuries. Constitutions outlast the governments, rights outlast products, and the institutions that sustain democratic life are also built slow. Sometimes, I agree, maybe too slow, but also speed without deliberation is not really the same as democracy. Meanwhile, AI, by contrast, is deployed on time scales of quarters, of product cycles, of funding rounds,

and this is a mismatch that is a real structural problem, in my opinion, of our current times. But I think here the open source open technology community is much better equipped to solve this tension because you care about long-term maintenance, you care about sustainability, and about projects that outlast their funders. Uh founders and whatever the model the democratic institutions depend on um on something that is built

in democratic time and not in on product the the technical decisions that we're making today can shape how citizens are governed for decades now. And then uh lastly, digital public infrastructure and sovereignty. Many of the benefits that I mentioned um depend on the digital infrastructure that now most of public life runs on. This can be identity systems, payment schemes, data centers, uh civic platforms. So, these are

the rails. And next to that sits digital sovereignty, meaning the ability to govern one's own stack and not be captured by a single vendor. So, the key question here is also not necessarily who owns this. There's many different models. Um public, uh private, or a mix of both of them, but the key question is whether it is built about democratic principles and whether we can still govern

it tomorrow. And here we say that needs four things: rights, rule of law, representation, and participation. I think this is a global we. I think we see what kind of impact AI systems can have. And um we sometimes make joke that the most successful products EU is producing are regulations. You look at this and how much impact AI systems can have on daily lives of every person.

I think looking into regulations is a good thing. We do not want that a few people, few organizations control, have that power to do this. So, we all are responsible to have this kind of democratic control. And I think the eight principles Ulana presented are very well resonating with the understanding, the self-understanding of the open-source community. So, I do not see so much differences here and I

think it was great having you on stage talking to us about this ones, but at the end I think we all hopefully all agree that this is valuable good things we should do. Thank you very much.