Making open source best practices visible for quality-driven regulated industries

This talk focuses on making open source best practices visible for quality-driven, regulated industries. The speaker discusses the need for compliance with various standards, particularly within safety-critical environments like automotive software development. They highlight the initiatives of the Eliza project, which aims to demonstrate that quality management can be effectively performed in open source projects. The session emphasizes the importance of systematic approaches to evaluate and document established open source development practices, considering the lack of quality standards that align with modern practices. The speaker shares insights from industry collaborations and the development of a quality rating guide to ensure that open source software can be reliably used in regulated contexts.

So then uh we'll give it a start. I I feel like with the hats hats on, it feels like being in a virtual web meeting, but just that you cannot switch off your camera. So be aware of what you're doing in the next half an hour. I will see when you fall asleep, for example. Um yeah, I call this make open source best practice visible for quality

driven regulated industries. long title. The basic thing behind is that um here here is a little CRA compliance aspons we're on the compliance track but there is a bunch of industries which ask for being compliant to certain standards some ISA standards and I'm mainly in the field of safety so we'll go through this briefly soon and the idea here is to say like can we claim quality

management being performed within open source projects Uh little disclaimer first. This is a work in progress. It's a initiative of few people meeting every other Friday to just discuss about this thing and drive it and you'll be directly pushed into the current work in progress. Uh we also always seek for more input. So it's nice to hear from you what uh you know as frameworks what you

would feel about things. So we come hopefully in a good interactive discussion later on. Let's see how it works with the heads up. I guess will work quite well. You should always get some trust originally in the one and see like who is performing on stage. Just briefly about myself. I'm an OSS community manager within EARS focusing also on automotive open source software processes which basically bring

means bringing open source to the developers to the company bringing company process into open source that's my main working field this initiative is from the Eliza project where it's a special interest group I give you a brief instruction what it is because Linux foundation we're here at Eclipse so you may not be aware I'm advisory board member I commit to the uh Eclipse escort project and I

bring a very long open source history. I think that's my passion basically uh also my friends and my wife typically will make some fun about me when I come with the next open source idea. Um quick thing about Elizar uh who is aware so raise your hand who is aware about the difference between safety and security half a bunch most uh minor recap as it's also being

recorded I don't know cannot ask the later on recorded virtual participants um if you see the pictures here we have the fire escape which is something which is added for safety but if you look at the top there isn't an open door. So it may not be as secure as you think because someone can intrude but at least you can escape in case of emergency. That's about

safety. On the other side you can see you can punch get your fingers into the door if the door closes. That's something which causes safety issues. It may be an emergency exit then it's really a safety issue more but at least there's a strong lock in front. So this is the security thing part just to have this one set up. And Elisa is about mainly the left

side but the idea about the lighthouse activities we're talking about come a bit from the right side because in open source and safety we typically hear oh safety and open source cannot go together. It's two different worlds. And if we like turn back the wheel of time 10 years, you may see that people say well security can only done by obscurity by uh your own development but

open source cannot be secure. So for this part uh we see that this changed until today and a lot of people explicitly choose open source solutions for being secure complying with cyber security and know that okay if I go to the latest version of some kind of packages so I will most likely be in the latest most secure version of the software. However, when you come to

a product development in regulated industries, they will tell you, "Oh, is this component certified? Is it compliant to a certain standard?" And then there was most likely someone taking the effort for certifying something and making it compliant to a standard, but then cannot keep up with the pace of the development in the open. So then you may be ending up with a security certified library from 5

years ago. I had just this discussion during lunchtime and where actually the better solution would be the library you take out today which you basically see in all the cloud infrastructure open source projects do it's just like setting up the scene uh yeah and in the LIA project we are looking basically into having a similar argument how do we can argue that open source software can be

safely used in critical environment uh it's a lot about dependability reliability we check this for various industries which also motivate this lighthouse uh best practices work group. Uh we work with industrial grade Linux distributors which work in all these industries. I give you a very brief picture on like who's involved in there and we also interact with a bunch of other communities not just Linux and we

really try to reach out in various field to get a good background and uh see that we always think in a systematic context right so this basically about it um here's a list of our members so you can see like uh top technology companies used to open source but also industrial bringing working in industrial fields of regulated parts And yeah, we just basically one and a half

years ago, we sat together during a workshop in the morning at breakfast. We all stayed in the same hotel and then we started this discussion and this was basically where where things originally kicked off. Uh yeah, so basically setting the scene to get you on board. uh traditional software development method speaking about the V model or automotive spice if you're an automotive CMMI uh very little rigger

is 90001 this is like built for proprietary software it comes from companies it goes into companies it's in company development and well open source behaves differently and there is currently no real quality standard I'm aware of you can correct me so it would be also a good uh which is a codecentric standard which really focuses on modern CI software development practices and really know how open source

community works but these are really driven on high quality and high efficiency and high structures protection and so on. So this is something which we see and uh still we see more and more requests by the regulator industry to handle complex use cases to go for collaboration to go into the open um but still there is a lack of matching standards. So we said the goal of

the project is to evaluate and document established open source development press practices. There have been also a bunch of presentation around trustable software framework. I mentioned it here briefly but this is also focusing on a product development giving you a trustable score. We said we do not write down the things we want to base on the best practice which we can observe but we don't want to

take this into the blue. So we then uh want to go for strategic or systematic approach in going into this and then come up with a guide a guide that helps you to see uh how can I rate the quality of something but that's just the extension of it. The base part is to really write down and to give a proof that what's happening in the open

brings a high degree of quality. Uh we have here three main phases and pillars. That's where we started last year off. So we were just saying like what is actually the status quo? How can we define these kind of practices? What do we find as evidence? Who made the little bit work up front to us so that we are not reinventing the wheel and then check that

whatever we came up as our assumption is being visible also in certain pilot projects. Then what we added up we're really at the beginning. I mean it's already April but we're not so fast as we could. We hope that for December time frame because we know some people from joint development foundation people are actually working in ISO standards um that they help us to prepare towards a

standard which could become an ISO something IC sen whatever we have we don't know um not that we really want it but we see that in the regulated industries they want to have a number if they don't get a number they're not convinced right if you need four digit if it's just three it's most likely not enough unless you're in aerospace then three is also a good

Um and yeah therefore we also look into existing standards. Maybe there is someone already working on this. Mary is something which is close. Um we performed some literature and research on this. We checked projects and during this phase we had this in the beginning. We gave a review end of the year last year and someone said like well have you actually checked the others which are doing

what you're planning to do because there are already a lot of best practices documentations out there. So we added this as an additional pillar. um try to bring things together and at the end end up to place it either as a project or bring it to communities and so on. Uh the poor thing about standards actually is like there are already so many standards out there and

well that's ridiculous. So we need to develop one more universal standard right to match them all. So we have another standard in there. uh that's really like the conflict which we're seeing that we basically do not want this but we haven't found something and how do we deal with it so uh we don't take ourselves too much serious so we have the XKCD here uh which I

think describes the situation quite well where we initially started off um we conducted a survey if there is actually potential interest and that's also how we found our members of the working group so we said like what is your interest which kind of industry are You are you aware of open source? Are you aware of standards? What do you bring? Uh we had a presentation on this.

This is recorded so you can later on check the link when the slides are shared and see what people thought about. We collected a bunch of people who said yes we are committed to working in this group and by this we kicked off. Um then we said like let's do not reinvent the wheel or do not duplicate work. So there was the automotive process thick group within

Eclipse. So we invited them, we joined also their meetings that we really make sure there is a big overlaps or there's no overlaps right depending on that we have synergies because it's a niche project and a niche topic so not too many people are in uh yeah and then we started we need templating on this to really make a comparison possible. So uh and continuously like today

also challenge the idea. It's good that we have a community base but we get always the best feedback from these kind of interactions. So we've started with an unconference at chaoscon at uh fostam last year. So this was important because they know budge about frameworks community health metrics and so on. We went with the beria people checking this to just check like is it worth to do

the survey. We performed the survey presented then the concept with the feedback which we got. We brought this into an internal workshop of the project. um then brought this up to the open source summit in North America presenting the feedback and all the results. Made a presentation again during the Lisa workshop brought this further into Eclipse meetup. So you can see we really try to work cross

organizational cross foundational parts yeah we like to go into a little bit of impressions uh where things are. So first we saw like we need some kind of structure though the structure things comes in with uh ISO 90001 actually uh why have we selected ISO 90001 we had participants actually from open source companies which doing industrial products and they said we went with our open source development

process into assessments for IS 90001 as a basic quality standard and this demands certain clustering ing like uh stakeholder management, ri risk management. Um and so we grouped these kind of things and said these are also areas which we want to assess when we look into a project. There is a sheet and I have the links as well. So you can really browse through the uh larger

spreadsheet go through the different parts. So and this was then at least a category but the category also need to have items like where do you check against you cannot simply say I check against how risk is tracked but we want like say what evidence is there so um we looked into the standard this was the second part where we wanted to look um the three main

work items which we took were um a re review paper from P Bernson on evaluation of open source OS for critical safety critical application. So this was a thesis. Sorry. Um the second one is systematic review. The systematic review I really recommend. It's not 100% up to date. So read it with a little bit grain of salt and a bit of care because it speaks for example

that um before you would test anything in the open, you would basically uh do a review. And if I look into current practices, of course, you would most likely run your CI/CD pipeline first with the linting and all the things before you take the first serious look into a pull request. So, so a little bit things changed in there, but it gives you a good hint and

it even tells you because it goes along a traditional V model. Also, it gives you an idea on what could be a requirement when it comes to open source project. What could be an architecture? What kind of testing is done? How can a project create traceability? And these are all things which standards later on will ask for but things may look different in open source. And then

on a latest one was also touch more on security as well is the the fevers from Stefan Touchna and uh this gives you just a more up to-ate 2025 perspective but more also security safety parts together and we used all this and uh filled up our axel sheet again with more and more of these kind of fields and um basically all said like we need a matrix.

We look on the one side on the process and we look on the other side on the evidence of what has been performed and the evidence is also there to see like on what we believe are certain practices and uh yeah this is a matrix so where we basically say like process robustness is there somewhere something defined this is the first thing there could be nothing defined

it could be implicitly defined that there are certain rule set that you need to follow. It could be explicitly defined with contributing guides or so institutionalized by really doing hard checks and other parts and having systematized things where you do uh governance in the project where you are talking like how to change processes how you add things why you add new things get this really in there

this would be like being excellent and then we have the evidence and we say like there's nothing a little bit like how which ratings are uh so going uh and then we try to find each of the parts as evidence really with putting in links. Uh we know the links can break but it's basically for others to follow up. So and then we also go through iterations

look like where can things go. uh we also will extend this by the pre-existing frameworks that's the new item which we work on and to show you also the like this two dimensions in practice uh this is a recent achievement we started basically with a symmetric drawing so like red part no was in the horizontal line and the first column and then the second column and second

row was with limited and also established was third sophisticated fourth and so on. So it was really a symmetric picture and while discussing this we saw like well wait not everything of this makes sense because it will be much better and our assumption is that evidence is higher rated than a process. A process is worth nothing if nobody follows the process. You can write down the most

beautiful process. So if you don't have evidence then it doesn't work out. So and therefore we gave a little shift and said like it's going up here. See? Yeah. So we see here this gets this move a little bit down to the left and here are some scaring lines. This basically tells us you can already have something on a sophisticated rating because you have things explicitly defined

writing it down but really enforce it by tools. This is like a rating which you can get. But some parts are not even possible. Like if you have nothing defined, it's nowhere defined. You cannot actually have a governance model. How will you have a governance model? Write down minutes and document your decisions. So you need to have something evidence and it need to be some more sophisticated

thing. insane part also well you have no evidence on the process but you really have a systematic way on doing things um I think there's so many things in the Linux kernel for example how they really handle patches and so on you will get a lot of feedback so they have systemized things and maintainers work on this work according to it so there is at least a

minimal evidence needed so it's not that there is no evidence it's just not properly documented and that's how we came to it and really went through all the cells and came into a very good discussion to say like that's where we want to go and this would be later on for you when you go in an assessment you can also make sure oh wait here's something I

haven't understood also this ends up in this beautiful spreadsheet again uh where you really go first on the left side with the areas this is from the ISO 90001 that you can have grouping and mapping the aspects are taken from the various standards or the various literature part and Then you do a subjective rating originally, right? So this is basically that you define it and it's of

course gets better uh over time and it's currently completely manual work and it's quite some effort to fill this kind of thing. However, we are also in the initial phase want to make sure that the left so the second column B where you we are going through the different aspects that these are really valid and one best practices which the people write down and you're here and

you know this asbombs are best practices right so check the kernel Linux kernel which is used almost everywhere it brings certain set of quality and see where the provide the recent cyclon DX aspx ready ass bomb download in their daily C eyeballs you most likely fail and we actually ended up with some of the project where we went into exchange because we cannot find always all information

so went in the community and I don't name it I don't want to blame a project but there was one project which basically said like what actually is an asbomb on the other side I mean if you have a small project which has a limited amount of code basically has one license does a versioning thing does regular releases, how complex will this ASPOM of this similar library

will be? Right? It's most likely so simplified because like well you may have an ASOM but it's not in an asbomb form as you expect and you may even trace your vulnerabilities and everything and have this listed in your release notes or so. So you do not require the need of an asbomb and don't need to touch like well everybody talks about CRA but go to the

open source people two years back nobody knows what the CRA thing is. it just came into open source like all of the sudden. So for this we went through uh the Zen project for a little bit. We went through Yakto project and always here it's a subjective first rating for us. So when you read it read it with scare and you can come to a different conclusion

you can also let us know. Uh we will extend for example with curl most likely because curl is also a very nice example with so many deployments so many downloads different or structures. So there need to be some kind of quality and like uh a kind of a little bit of dictatorship with the mono maintainer in there. So we want to check to these parts and go

on with this and yeah here here you can basically see like the rating 314 colorcoded uh reworking this continuously. Then uh the latest part was on existing records of best practices. So we'll continue and this is really something which will pop up in the next month. So we will get the first evaluation and the first feedback in June. We again go for a workshop uh from the

project and then bring it most likely in September October frame to an open source summit wherever we go to a conference where we can challenge this in a wider audience. and yeah we will look into scorecards insights trustable framework which is in the clips. I know that the Eclipse people would like to see Eclipse up front. I missed this. Sorry, it's Eclipseible KS community. Uh we'll go

to Don Foster, I guess, exchange with her again. Um and it would also be not a show list soon. So if you come up with new ideas, uh it's a little bit wider than you may expect. Uh so here you see we have of course our activity, right? This is one of them. uh but we found open ops already brings like three different kind of judgment criteria

for uh best practices chaos has evidence on forest or dashboards and so on. uh there's an insights tool, there's the uh batch I think I haven't mentioned them because I couldn't find the real evident on the process but there are the Eclipse maturity batches as well Apache brings it CNCF brings it the to-do group has guidelines on how to do open source properly. So we really would

like to see as this and yeah we we figure out that when we look into this we have actually no clue on what it is meant for who's the target group of this what's the effort to perform it is it complete and that's basically where we now look and so we group them we cluster them into different things like is it meant more only for security how

much is the scope of it where does it go to uh we'll sort this out and then short list and we'll most likely say like based on this we have evaluated this this is a reason for going forward or not and then looking very much into details we definitely will know that we will look into the uh scorecard from open SSF and we will also build something

on the LFIX insights because we tackled this already end of last year and while we were sit there suddenly suddenly a new not so often joining community member showed up and said and she said like you know What about standardization parts which also deal with these kind of things. So you see on the right side there's the NIST part uh amit tag which is like on marine

software quality which is specific but this is something which we missed in our standards check and so on and we don't know it's more like the stretch part of it. It's not exactly like open source best practice but we at least want to assess as far as we can if it's available in the open how does it fit and then like how close the connection is you

can see with open chain right open chain also brings in certain things you should do and this also gets into an ISO standard so an ISO standard IC standard is not always completely disconnected from an open source environment so we will look into this as well uh and all this which I've shown basically uh historic did to be processed in two working drafts with the links here.

Uh so you can either see this comparison which is super early draft and you can see all the practices a little bit of the ratings. Everybody should have read access. I'm not sure about commenting but whenever you go on this and say like request commenting rights access it typically takes most likely a day or so when I check again and then I give access for editing or

commenting at least. So this there and if you get other issues uh you could also approach directly. The idea is uh yeah I also like to get more participants in there. We have a core group of four to five people working on this. Uh we have an extended group of 8 to 10 people joining the calls. There's minutes of everything there. And to get involved uh it's

every other Friday 12 p.m. till noon time on UTC. So this universal time uh we have the mailing list. So mailing list is mainly there to announce the meeting agenda and make sure that the meeting happens. Sometimes there is some exchange but we have more like ad hoc exchange on discord and yeah you can see also read the vicki and actually the main repo is not too

populated because we have still a lot of work in the in progress we consider to get in a landscape that we can all see like which project has been assessed and so on. If you actively want to contribute uh there's an in-person workshop in London happening uh June 9th to 11th uh it's free of charge uh but you really need to later on confirm because we have

limited slot it's workshop it's really interactive it's much more topics around safety and other parts and there's also virtual participation possible if enough people mark their interest in virtual because it's never as good registration is open And here as I targeted to have the discussion to get feedback and potentially get out enough time for the next speaker to speak up. I'm already at the thank you part

because well I hope for discussion I hope it will work out with the headsets. So if someone want proposal like this would be a good uh framework or this is something you missed. Have you considered that this would be really kind of you or just ask questions like whatever you have in mind. So anyone comes up with some question any question we have time. If not, uh,

thanks very much, Philip, for the presentation.