From ethics to evidence: Operationalizing AI governance in open ecosystems

This talk focuses on the importance of achieving compliance in AI technologies and the necessity of contextual understanding in assessments. The speaker describes the new company they co-founded, which provides consultancy and products for ensuring compliance with AI regulations, notably the EU AI Act. They emphasize that while documentation is crucial, it often fails to capture the actual behavior of AI systems in practice. By discussing an experiment involving AI models, the speaker illustrates how minor changes in configuration can lead to significant shifts in outcomes and urges the integration of behavioral testing into standard workflows. The aim is to transition from mere ethical considerations to concrete evidence of responsible AI deployment.

Why am I here? I hope you can read it. If not, I have to talk out loud. Uh, it's my first OCX, so I'm not necessarily from the open source community, but me and a colleague, we just started a new company. Uh, let's say two companies basically get the first time. One is let's say a tech compliance consultancy based in Berlin and the other one is a

product that we built for professional services in order to do their services in a compliant way. So why from ethics to evidence? We are right now, you know, helping a couple of of of customers mainly in the German market with their AI activities. When it comes to how do we stay compliant, how do we get compliant? Is everything we do with AI, you know, under the EU

AI act or not or what shall we do? And as I'm in Brussels, of course, I have to mention the AI act and also I will complain only a bit about it. So my co-founder who as said we are a new company had to leave Brussels yesterday. So I will also cover his parts today. So why from So I would say when we talk about governing AI,

AI compliance and other topics, you might see a couple of terms that are used in your companies, right? People talk about checklists, people talk about usage policies, about risk assessments and so on and so forth. And I think while all of this is pretty valid, we believe there's one thing missing. Why? Because documentation describes the intent, right? I mean, we just heard a bit about it and

I agree with many points of my of my colleague that spoke before. Documentation is important. Documentation might be required under certain regulations, but what it's not describing is typically behavior. So, what you see is it captures the architecture, it it captures the intended use cases and so on. But what's basically missing is always the context of how and where it is running. So what we did for

let's say a bit of our own research but also to explain the importance of context to customers is we took an experiment that was done in 2006 with a series of judges. There was a group of judges. They were all giving the same case to judge upon and in one group there was like a note that the average you know sentencing time is 9 months and for

the other group the average sentencing time was 5 months. All judges have been told to ignore that average but what happened in the end is they just by that piece of information alone got biased. What does it mean? The group that got the higher anchor in the documents provided on average gave a higher and the group of judges with the lower one even while being specifically told

and this is humans we're talking about right now ignored this information they still got biased and the average sentencing was lower. So what we said is like huh what happens if we use this experiment on AI. So what we did we do we took two models model a cloud haiku 4.5. Why? Because it's cheap. So and on the other end we took deepseeek version 3. So they

were fed with the exactly same information. It was configuration. It was a YAML file. So they got the exactly the same thing and were asked to sentence on the on that case. What happened and it's really hard to see those numbers but what you can see is like Claude Haiku had a very low anchoring effect meaning their results less biased so to say only by 8.3%. But

Deepseek had like an anchoring effect of 41%. meaning again just with those two data points being different uh an AI model gave a completely different result >> what did we do to fix this or basically one step back if we look at those two models now and what they provide they all give you model cards they give you proper documentation let's ignore the fact that D6 is

from China but in general documentation is there a trust center is there the policies are there basically We we had a couple of German manufacturing customers that said Deepseek basically covers all our governance and compliance standards. Why? Because the documentation was there. When we showed them what happened here, they were like, "Huh, something is missing." So to show you, for example, one thing that's changes a lot,

we added one thing like a a chain of thought in the original prompting. We said think about it twice. It was really a very simple one. I can show you if you want the YAML file afterwards. A really simple chain of thought uh addition to the original prompt. What happened with DeepSeek was the accuracy jumped from 63% to 97. So just that one little change in the

configuration made it behave completely differently. And as said we are in Brussels, right? So AI act written just around the corner. What do they say and what do they ask for from you? They say you has to have to use the most appropriate and targeted risk management measures. It's still up to interpretation what what that is, but with the information you've seen before, you might want to

think about are written policies and is written documentation enough. It also said no you have to test for accuracy for robustness for cyber security not documentation is not just the only thing that's needed so most of the regulation and it's not only the EU it's like it's in Singapore it's in Australia wherever you read the local regulation they typically demand behavioral evidence and that's what most of

our customers but also what most of our market what we see on the market. That's what companies are missing. So, the analogy I used was, you know, you're checking the menu, you're not tasting the food. I mean, who would judge a restaurant's quality by just looking at it from the outside? Maybe some. I'm not I'm not sure, but I want to taste the food before I say

this is a good restaurant. This is maybe not the best restaurant. And to give you as an a lightning talk, how did we do this? We had a simple uh testing process in place where we said it's one configuration file that's always the same. It's integrated into our CI/CD. And what this gives you is it has an hypothesis as code not written somewhere but as code. It's

automatically executed with every run. It's not like your compliance officer checks it every three months. It's in every run. It's one piece of your development process. you define passing and failing thresholds. So if your accuracy drops below 95, stop the build, right? Not not ship it to your customers and it's version controlled and so on. So what we see as the next step happening and too few

people talking about these days is extending your Not making it more complex or more you know workheavy but after you have tested the piece of software you you want you built and you want to ship run behavioral tests and they can be quite easy right doesn't need to be complex from the start and that's what we want to say in those 15 minutes is really you can

start doing this tomorrow after you left Brussels what do we say pick one model you use pick one bias you want to check up on. Let it be accuracy, let it be framing, let it be bias, find one experiment as we called it, run this integrated into your workflow. So for us, what we always tell our customers, if you say you are governing AI and you don't

run behavioral tests, you're not governing it. You're just producing with your assumptions. It's not tested yet. So for us, the idea is how do we get from ethics, we want to do the right thing to actual evidence that we did the right thing. Thank you very much. Any questions? Yes, please. I'm really out of the domain I would say but um I had some experience in uh

um behavioral driven development >> and I know what's the scope under test in AI. I don't even know what's the human under test. So AI is I think a much bigger problem and I wonder >> how do you um select the the biases and the expected behaviors in such a way that you are good enough confident that the AI works. >> I mean that's the question I

often hear and it's also a question I heard even before AI. That's what I typically say. I mean I'm in the like tech scene in Berlin since 20 years. So you know I've been in the in the DevOps wave, I've been in the cloud wave and it's basically the same question for me. It all starts with the process being done start to end. So we had a

couple of discussions these days where it's like we have to start using AI also when we come to product management to requirement engine only to coding right if you code you just produce a lot of things without any context. If you start the context from where the requirement comes from, you start documenting what are the biases we want to check upon. If you just build VIP code,

a new VIP coded CRM, how do you want to check for any biases? You can you can guess, you can take best practices. But for me, the the solution in a way is integrating it into your whole development life cycle. you know have the vertical cut and then build out AI horizontally. So we have like one one example it's like a German automotive company and they have

a financial services arm and they have a digital unit. So what they did is as they are a bit more flexible than the big old car maker. They built out a new development process from start to end with a very limited, you know, scope of II in each step and then scaled it as they went along. But the one thing that changed is, huh, if we want

to test in the end if what we ship is compliant, is nonbiased, is ethically safe, we have to say that at the beginning. And it's hard. I know when VIP coding is so much fun sometimes, but if you really want to ship it in the end, there's no way around it. Any more questions? >> Thank you, Tima, for your presentation.