About this talk
This talk discusses the concept of trustable software in the context of modern, software-defined embedded systems, emphasizing the challenges and risks they present. The speaker, John Ellis from Code Think, outlines how the complexity of software, which spans various industries such as automotive, medical, and aviation, has outpaced our ability to ensure its safety and accountability. He introduces the Eclipse Trustable Software Framework (TSF), designed to provide measurable foundations for engineering trustable software through six tenets: providence, construction, changes, expectations, results, and confidence. The framework aims to foster trust within the software development community, making the processes for validation and evidence generation more open and collaborative. Ellis calls for an end to the opaque certification methods of the past, advocating for continuous evidence generation to bolster trust in software production.
Full transcript
[music] >> Uh come on now. Let's try it again. Good afternoon. All right. Thank you very much for choosing this over the many other options that you had uh and joining me on this quick journey uh to discuss trustable software uh open source and open accountability. My name is John Ellis. Uh I'm with a company called Code Think based out of Manchester, UK. Don't you can ignore
the accent. I I truly am European. Um and I'm here today to actually take you on a journey uh about trustable software. So, let's begin. Um architects are known to have said sleep under the bridge the first night it opens and we can trust that which we have built. And the idea was that if they wouldn't why should mankind humanity trust crossing the bridge? Today, our embedded
systems are very much software defined. That is not a surprise to this audience. But it is a surprise to your executives to your politicians likely to your family members. Vehicles medical equipment transportation systems broadly logistics supply chain even airplanes. Everything is software defined. Unfortunately, the software code that is in that definition has outrun our ability to reason about the systems. We have hundreds of million lines of
code in a modern car. Modern systems easily. We have thousands of components. You might build them. Your partners might build them. You might pull them from open source. And for those of us who operate in the regulated industry we are expected to be able to keep that which we have shipped safe and operating for some definition of and operating and secure for anywhere from 15 years in
the automotive industry to 30-plus years in the industrial and factory industries. In essence integration of everything here has become the moment of truth. How many of us have sat there with our sphincters tight as we push the button? Oh my god, please work. Integration is where reality happens. Where we bring suppliers the reality of where code comes from. We bring it into a system and we try
and reason about it. We try and construct it. We can try and bring it together. We try and hit an SOP start of production date. And often times with the best of intentions, we are forced to forget about anything other than start of production. We must not screw up the factory. We must hit our ship dates. So, engineers are left to not reason about the ability to
worry about SOP and 5 years later or 10 years later. And what happens in that integration moment is unknown dependency surface. We find regularly non-reproducible binaries. Oh crap, this built yesterday, but I can't build it today. Or I can build it today, but it doesn't look like it did yesterday. I don't understand what happened. Where did this piece of code come from? It wasn't here the other
week. What do you What do you mean you need evidence, Mr. Regulator? My code works. You should just accept Software is moving at highway speed. It's moving fast. And unfortunately, failures are propagating faster than our ability to investigate them to explain them to prove and remediate them. The result of this is an estimate that is measured in trillions of US dollar in terms of economic loss. In
light of that absolutely staggering situation you have to ask yourself why do we trust software? Would you sleep under the software that you ship? Because quite frankly and honestly, all are guilty of not sleeping under our software and not being willing to sleep under our software. We have the tools by which to try and bring accountability to the systems. But what we don't have what we lack
is trust. We have S-bombs, software bill of materials. We have continuous integration. We have updates. We have requirements. We have tests. We have assurance. What we don't have is trust. Everything we have is fragmented. The result being that companies are left alone to try and traverse the waters and figure this out. And the response that I am offering you today that the Eclipse Foundation at Code Think
are bringing forward today and we've been bringing forward over the last 18 months is what we call the Eclipse Trustable Software Framework. In short form, it's a structured measurable foundation for building embedded software that can be trusted across open source, closed source, OEMs, regulators, Europe, China, Japan, the US. And what's important and key is it's an open source endeavor. The framework is simple. Six tenets of trustable
software. When we're done with this session, these slides will be made available to you and to anyone else you want. You can also be directed to Code Think or the Eclipse project. And what you'll find is some history there. These tenets didn't just magically appear. They're a result of very thoughtful thinking in an open source project itself almost 10 years ago. The result of Bosch asking the
question, could we actually use open in a proprietary mechanism for safety? Could we use open source to deliver safe product? A group of people came together to start talking about, well, what does it really mean to trust software? The result of that is in fact these six tenets. Very simple. Providence. Where does it come from? Construction, changes, expectations, results. And conclu- uh confidence. The goal is that
TSF turns trust this ambiguous word into an engineering discipline. It's measurable. It's repeatable. It's reviewable. We can We can reason about it. The six tenets. Providence. Know what you are building. Be able to actually trace every component to origin. Be able to capture supplier and open source or and even closed source dependencies. Make the supply chain inspectable. Often times we find that while this is very easy
to talk about, it is very, very hard in practice. How many of you have actually tried to do hermetic builds? Construction. Deterministic and reproducible builds. For those of you who think reproducibility is not possible please come and talk to us. It is possible. It doesn't come for free. It requires work. But we can in fact make something reproducible. And when we do that, we now have confidence
in our construction mechanism. We have confidence in the tool chains and the environments. We have confidence in the factory pipelines that generate and construct the software in question as well as the evidence that accompanies Changes. Controlled change over decades. How many of you have ever lived in fear of touching your software? In fear of touching something that's already been released. Of having to try and convince management
to touch something already released. That right there is key evidence that we have failed. We should never be afraid to change that which we have built or and shipped. We should be able to reason about that change. We should be able to explain that change. We should be able to reverse that change if necessary. The release integrity concept is continuous. Minute to minute, day to day, week
to week, month to month, not once and done or once every year or once every regulator inspector for the certificate that I received. Define what good means. What does it mean to have good software? What are the expectations I as an engineer, I as a company have about that which I am releasing? Whether it's regarding safety, security, resilience, reliance, performance, it doesn't matter. But what are your
expectations that you're making claims against to your company, to your shareholders, to society at large? Have very explicit requirements and constraints and very clear acceptance criteria for the evidence that underpins the expectation claim that you are making. Results, simple. Prove what the system does. Test verifications, right? Evidence measurable, evidence bound to specific builds. More importantly, because it's software, not just prove what the system does, but prove
what the system doesn't. Verify that the system doesn't do what your assumptions are about what the system cannot do. And prove it every single time you hit the return key to do a build. At the very end of the day, confidence. evidence do I have that allows me as an engineer or as a company to reason about the trustability, that which I'm about to release? How confident
am I that it will not bring harm to goods or people? destroy the enterprise value of my company or those who operate in my ecosystem? How do I support the regulators, assessors, politicians, society that how I am reasoning about my confidence, they too, independently, can reason with me? And then ultimately, how do I collaborate with other ecosystems who might want to work with me or who are
impacted by what I have done? As you sit here, you're very likely saying, "Well, yeah, John, that makes perfect sense, of course." Unfortunately, it's not as easy to do, and we know this. And so, the Eclipse project is bringing forward this idea of trustable by design, including a fundamental construct about how you actually build and deliver what we just talked about. The idea of construction, the idea
of factory as a factory, where we treat the idea of software as equally as we treat the real world of trying to build and construct at scale and speed. And the key shift that we're trying to get across and that we are advocating for is that evidence is generated continuously. My anecdote is every time you hit the return key and do a build, you should be generating
the evidence necessary that matches, supports, and validates the claims that you are making publicly. You do not assemble the evidence at the end and hope that your checklist exercise is complete or that the assessor is asleep and misses that you're missing so much of your evidence. We are advocating in public at the Eclipse SDV project, you heard it this morning, a single integration mainline concept. This is
an anecdote for discussion, but this is an idea. And what we are showing here is on behalf of this project, Code Think has actually demonstrated this in an enterprise at scale for a company trying to deliver safe software to move a vehicle in space at 80 plus miles an hour, 100,000 lb, and no human being involved. It is doable. It is hard. It is engineering. But it
does work. More importantly, it does matter. To be clear, TSF complements. It does not replace how we have historically grown up with our existing standards and our tools. In terms of standards, it is cognizant of and complementary to the functional safety standards that exist, the cybersecurity IEC or ISO, the quality and compliance demands that we have. In terms of practices, it doesn't introduce anything that good software
engineers already know about. Reproducible builds, software bill of materials, provenance, the idea that I actually can make a claim, prove the claim, test the claim, validate the claim continuously, and be able to show it to you in a way that you can consume it. And in terms of outputs, traceable releases, reviewable artifacts, assessor-ready evidence, these words, they're everywhere. Also part of TSF. Nothing there is new. Trust
is a multi-party problem. The regulators and assessors, generally, we have found as Code Think, and again, if you go see history, we've actually gone through this. We have been assessed at multiple different standards, 26262, 61508. And what we have found out is that the basically don't trust you. And until they can trust you, the engineer, the engineering community, the engineering they are loath and reluctant, no matter
how good your checklist is, to actually say yes. Trust has to be built. The OEMs have learned and built trust directly around an ecosystem of assessors, regulators, testers, companies. The suppliers and then the open source communities. Collectively, we all distrust each other with good reason, but that becomes friction that we need to eliminate. And so, our goal for the Eclipse Trustworthy Software Framework is trust by evidence
at ecosystem scale, not enterprise not country scale, but at the ecosystem scale in which you operate. Now, everything I've been saying, you're like, "Yep, that makes sense. I'm good. I'm not really learning anything new, and you're standing between me and lunch. So, tell me, John, why I should give a damn." Here's the traditional certification approach, and I don't care what kind of cert you want. ISO, IEC,
safety, security, quality, I don't care. As a company or as an entity who has something that they want to get certified, you are a company seeking certification. And in the world that we operate today, you call upon assessing companies. Some of them might offer consulting services, but these are the companies who through ANSI or other certification bodies are empowered to bless you. To actually say, "Yes, I
have seen the evidence, and yes, it is good enough for the purposes of whatever the standard is that the company is seeking." And that's the key word, whatever the company is seeking. So, when you go to an assessor and say, "I want my product to be assessed," their first question is, "What is the standard against what do you want us to assess you?" Company chooses it. And
then the assessment company works with you, potentially consultatively, but ultimately, it assesses you. And it continues to work with you until you are certified for that particular product. What is key about this effort? It is governed by contracts, managed by confidentiality agreements, and absolutely, completely opaque. It explicitly prohibits the ability to reason about what you have just built and what you're trying to deliver. The world, society
at large, is left to trust the assessor. And I'm sorry, but the trillions of dollars of loss should give us pause for why we should put that trust. So, what are we suggesting? A brand new approach where the TSF is in fact what you look to, and it has the mappings for the industry standards that you seek. So again, our company is seeking certification, and they will
still go to their assessor, Exida, UL, TÜV, and they will say, "I want to cert. I'm going to go after the TSF. I'm going to use the word Exida right now, cuz that's somebody we've worked with. And Exida will say, "That's great. You want to assess against the TSF. What standard are you actually looking to go against?" I'm going against 61508. They're like, "Great." Previously, they would
have done something under contract and confidentiality agreements in a proprietary way to get you from point A to point B. Today, in open, they're now going to take the standard and the mapping, and they're going to say, "Let's work on it and get you from point A to point B." And you as a company are actually assessing yourself and certifying against the standard, but through the lens
of TSF. to actually deliver on that promise, about 3 weeks ago, Code Think announced that we were in fact doing that. We were taking a Linux-based distribution through working on a 61508 cert. Because of this journey, we have a baseline safety assessment that says our Linux-based operating system can be used in properly engineered systems up to ASIL D, SIL 3. The TSF practice itself has been certified
with Eclipse and Exida. And the return on report is the TSF induces and guarantees a software development process that is an excess to what is required for SIL 3 development. And so we've taken all the mappings and we've made them available under early preview. Exactly the mappings that we are going against for our certification efforts. And what is very cool and what our call to action is
and what we've been working with the Eclipse Foundation, whether it's the ORC, the STV project, or others, is that this works for any level of mappings. Tell us your mapping and we want to work with you. To be clear, the only mapping today available is 61508, but it is open, available under an early preview release, and it will be fully available, fully downloadable, fully usable, fully reasonable,
once we've actually gotten our control OS through. And so what exactly have we done? What exactly is available? Well, it maps the TSF, all those tenants, those six tenants that we went through, it maps them 61508, what the expectations are. So where Exida or TUV or UL would have had a proprietary mapping, likely in spreadsheet form, this is now open, public. It is available, it is digital,
it is available and consumable, and it's reasonable. We've given evidence templates, our providence, all of the different tenants for how we show up. And we have built it from real-world integration, a real absolute effort to bring a Linux distro through so that you could work safely with Linux at a SIL 3 or ASIL D level. So how can you use it? You start today. You can go
under the preview and go get it. And and you can In fairness, TUV and UL are aware of it. They haven't yet adopted as quickly as Exida, but Exida is very much aware of it, and Exida absolutely would understand if you went to them and said, "Hey, I want to do something with TSF and 61508." They would know exactly what that meant and how to work with
you. You can use it right now to start aligning your tool chain or CI/CD. It aligns. And then more importantly, we want you to bring feedback. We are operating this as an open source project. Multiple different avenues. There are stand-ups, sit-ups. There's a birds of a feather today in a workshop form for some of the changes to the trustable graph that we've learned about. There's a call
hosted every second week or every other week where we talk about in depth what has to happen, etc. This is an active project at active and you're invited to participate. Now what's really cool as we sit here today, as you heard Mike earlier talk about ORC in the CRA, it's coming. It's really coming. And while we at Code think started this trustable journey almost 14 years ago,
and we started it from a journey of what we have come to realize and now continue to begin to advocate for and evangelize is that trustable tenants one, two, and three, providence, construction, change, they are universal. They're not safety related, they're a foundation for safety, but they are not safety. They are universal. And guess what? The CRA requires them. So when Mike was talking about the attestation
process and the due diligence process, when we are talking about how does a company actually comply with the CRA? What we have been saying in the code and compliance conversations, what we've been saying publicly is the CRA tells you what. Tells you what the EU is expecting, what the commission is expecting, what they on behalf of society is expecting. And we at Eclipse are proposing that the
with the CRA mapping tells you how. Shows you how. And helps you deliver the how. So then I ask you, if you would not be willing to do this, and you work in the automotive arena, then shame on you. You should be willing to sleep under your software, especially if you are expecting society to do the same. I welcome you throughout the course of today, multiple TSF
sessions, a workshop later this afternoon. Many of my colleagues and friends will be on hand throughout the entire week, available for questions. As I said before, these slides will be made available publicly. I thank you very much for your time, and we'll open it up for Q&A. >> [applause] >> Oh, come on. You got to have questions. I was provocative on every slide. You got to have
questions. Yes, sir. Yeah, thank you for the Oop. Yeah, it's on. Yep. Ah, yeah. Thank you for the great talk. Thank you. Uh so you're talking about certifying a Linux distro. What about a Java-based application? What about Java-based? Yeah. So the So the original genesis of all this, I will be very upfront, right, with an embedded arena, highly regulated, with some very specific sort of characteristics. Um
when Code thing came to Eclipse almost 2 years ago and offered to join and then offered to bring all of our work and then do all the continued work in public. One of the outstanding question was, what about Java? What about other projects? What about Rust? What about other languages? This is language independent. It is It doesn't require C. It's not dependent on Rust. Where you will
see things in providence is, where are you getting a package So where did that Where did Where did lib C come from if you're using C? Where did your Java package come from? How are you Do you understand your dependencies? Are you using an open VSX pull and do you understand what pulls, frameworks, etc.? We don't dictate a language. We don't dictate to you what language you
should use, and we don't dictate to you how the language that you're using should or could be certified. That's not what the project's about. Now if in the course of doing Java certification, there are things that we can learn and make available in open public to make it easier for people, then there are avenues for doing that. But those are avenues to make it easier, not to
dictate what should be done. So I understand you're just recompiling the whole the whole stack, the whole dependency tree. Uh well, just not just, but you're are recompiling the whole dependency stack. Uh companies what? Is this I shaver? Sorry, I what was the question? Well, yeah, what you're I I'm not following your question. Ah. Um to be able to reproduce some dependency stack, you need to recompile,
basically. Mhm. True? So for a Java-based application, to me, as far as I understand, this would mean to get rid of Maven Central, uh NPM repositories, whatever. Okay. A- Again, so there are nuances and and specifically about the Java, if we have to recompile, like there are specifics of language that would be represented in providence, well, very specifically in construction. we're not worried about that today, but
we are acknowledging that there are language uniquenesses that would themselves bring technical challenges that may not have been considered as part of TSF development. And we are encouraging people to come either contribute or raise the issue or ask for. I mean, Eclipse as a historical entity, right, came about because of Java, came about because of the IDE for Java, right? So it is not lack of understanding.
It is what's the biggest need, how does it align, and how do we get through? But there is an absolute understanding that it would be great if all 425 or whatever the number of projects could in fact use trustable, which would mean we'd have to address what does that mean for Java, etc. Okay. Yeah, thank you. Yep. Other question? Um I took a quick look uh before
the talk on the the GitHub repository, and uh what what I asked myself, so all all you you've described on theoretical level, that's clear. Um but what does the the project actually provide? Is it providing tooling? Is it providing descriptions how to map things, to do things, best practices? I've seen there is some some Python uh in in the project. So how would this look in reality?
Yeah, so if you're talking about the TSF project at large, what does it What does it really provide, right? A few things, and I don't have a slide on it, so let me just wax poetically for a second here. Um first off, it provides a community and a community of people who are going to reason about software at scale. Secondly, it provides the language for that reasoning,
and the ability mathematically to reason at a level Three, it does provide definitions for each one of those tenants. It describes what the tenant is. It deconstructs that tenant as far as we've done the work to date against what we're trying to do, which was to bring a Linux-based distro into the world at SIL 3 or ASIL D. So, the idea of Providence against an open source
project like Linux. Great. Where do things come from? How do we build? How do we construct? How do we do reproducibility? It brings the reasoning that we Code Think have done previously now in public for why certain choices are made. Systemd versus init D. How do things work? Et cetera. And the reasoning for >> In what form do you provide this reasoning in form of >> all
it's in YAML. Sorry. So, the TSF graph has a whole bunch of things we can follow up afterwards, right? All of this is readable, machine readable, human readable, consumable, understandable, allows for you to reason. Yeah. >> Okay, that's great. >> Yes. And we can talk more about where things are at in the project. And if you come to the workshop, like we're going to go through a
bunch of stuff at the workshop. If you can make it, that would be ideal where we can go through it cuz again, we're going to even have some of the engineers from the team on available. Thanks. Yeah. Um I was wondering, can I do some kind of self-assessment based on that framework? And how valuable would would that be for an open source project? Or do I always
need these intermediaries, these assessment companies? So, the question being would it how how could you do it could you do a self-assessment and how good or valuable would that be? Do you need to use the you know, assessing ecosystem? Start from the beginning. You can You can This is open source. You can deploy this and as a company if you're willing to stand up, understand it, and
reason amongst yourself, and want to do it, you can. You're a project and you want to do The question that would be in front of you is the cost-benefit for trying to do something of this nature. Now, do you have to use an assessor? No, we're not saying that. Not at all. We're just pointing out that there exists an assessment community. Companies have used them for years.
They've got trust with those assessors. So, what we're suggesting is that there is a way for those assessors to continue to deliver on that trust, but in a very different way, not in this sort of confidential way, but now in a very open way that allows reasoning at scale. In terms of open source projects, there's a current rich conversation happening at the CRA for how we make
the ability for a project to do attestation or due diligence in support of if they wanted to manufacturers. How is this world supposed to work? This is a very very rich conversation. Is there a world where maybe they get paid that a project chooses to say, "I'm willing to make a very small subgraph related to my project that I stand by that you could consume in your
trustable argument later?" All of these are totally doable. They are technically not prohibited, but what is lacking is the business and commercials conversations around them. And those are what's happening at ORC and coding compliance, et cetera. So, again in summary, yeah, you can do self-assessment if you want. This We're not trying to say any of that. We're just trying to stop the opaqueness, the confidentiality, NDA-based. The
society demands more, and it should demand more. And that's what this is intending to Thank you. Yeah. So, I see Johannes Kepler and Johns Hopkins University are participating. I was wondering if there's a larger higher ed or education engagement. I'm assuming those two are probably working from projects that are coming out of them rather than from an institutional level. So, looking at this, I see the industries
were automotive. I forget what all they were. >> Medical, aviation, anything that's highly regulated. Yep. >> Um well, education, higher ed being highly regulated, I wonder if there's activity around standards within higher education, student data, all those sorts of things. Um and then second question, um the OSPO Alliance, following up a bit on this question about using this as a tool to assess your your own projects.
Um is OSPO Alliance at all involved with this? Let me ask second question first. Octo Alliance, I have heard of it. I'm not going to I'm not going to overstep myself because I don't know enough about it, but I believe that they are they are at least involved at the on the on the adjacent side of ORC, but I I I they're not directly dealing with us
for TSF, but that doesn't mean that they're not aware of it or not participating. So, I don't I would I would you hold that In terms of education, I'll break it up into two. In terms of education as an industry and being {quote} certified or regulated or whatnot, we have not thought about that. Um and we would welcome if you could come to the workshop, even if
you just come for a moment and say, "I'd like to open a project related to education." Please come. But what we have done with Carnegie Mellon and a few of the other universities is as part of Providence, we've begun to realize that supply chain understanding is grossly lacking in software related to embedded or procurement or others. There's a There's a just a gross misunderstanding. And most companies
have this belief that if I have a specialist in the physical with one class, they can become a digital specialist, whatever that might be. And we're trying to create training programs and classes. Western Kentucky University with their supply chain program, Carnegie Mellon, and others coming together to have a conversation about like, "How would we teach this?" Because it is a legitimate question. If this is so good,
why aren't universities picking it up yet? Why isn't it being taught in engineering classes, right? And so, we're in this catch-22 early day, but we are aware of that arena and that thread. Um University City London, uh London School of Economics, the digital sovereignty movement that's happening here, right? I mean, a bunch of stuff from an academic arena is now starting to surface saying, "Okay, how do
we play? How do we participate?" Both from how do we teach it then how do we how do we use it as a tool? So, I I would welcome more conversation. Afterwards, I'll give you the links to the people or to the professors. Yeah. Any other questions? It looks great. But what about adoption? What adoption within Eclipse? It is connected somehow to the governance principle. And which
main Eclipse projects are already using or are planning to adopt in the framework? Yep. So, the adoption question is an awesome question. Thank you for it. Uh there are three answers to you. Within respect to publicly, there are companies publicly that Code Think are working with that we've announced with who are using Trustable to deliver in automotive or transportation space. Um for the purposes of this recording,
I won't talk about it. I can talk to you later, but I can point you to public announcements. Uh within Eclipse in the project, TSF from the beginning was envisioned because we were coming from a safety arena. And again, this is pre-CRA and full deep understanding. So, we put it inside of the SDV project. So, the SDV project is in fact using it. You'll hear something from
Daniel Krügler later from ETAS who's talking about, "How do I take this and bring it in projects that ETAS are trying to do?" But more broadly, the Eclipse Foundation is now aware in the ORC, so the open regulatory um that the TSF could in fact be the how that companies or entities deliver against the what that the CRA asked for. And so, from that standpoint, and it's
only been maybe the last nine Well, since August where we did our very first idea, we did some mappings initially, we invited people to start the So, it's housed today in SDV, which is using it, but it is now looking seriously from an ORC perspective for the how against attestation or due diligence. Any other questions before we let you go for lunch? All right, ladies and gentlemen,
thank you so very much. I appreciate you. Please come see us at the uh workshop and come see any of the other Trustable stuff. I'll be around for questions here. Thank you.
