openDuT: The open source revolution for distributed ECU testing

This talk focuses on the open source revolution in distributed ECU testing, highlighting its significance in facilitating ECU development and testing globally. The speakers outline the existing pain points in ECU testing, including logistical challenges, high costs, and integration issues arising from the complexity of modern ECUs. They introduce OpenD, an innovative framework that enables seamless transition from virtual to hardware-based testing without major setup changes, allowing for dynamic integration and remote communication. The session also discusses the architecture of OpenD, its components, and the use of tools like Viper for testing. The speakers emphasize the importance of collaboration within the open-source community to enhance this project and its potential applications beyond automotive to other domains.

[music] So hello everyone and welcome to our talk today and we are excited to share our talk open the open source revolution for distributed ECU testing and as mentioned why it's valuable to be able to test ECUs anywhere in the world no matter where you're sitting no matter where the ECU is um for that we first uh have a glimpse you on the pain points the ECU

development ment and testing currently has what people currently do and how open do is fitting to the gaps what we currently have. Afterwards uh we have a clo a small overview about the components we use. Then we make a deep dive in the architecture and security concepts to show how you can deal with it to uh make a secure ECU to EU communication even in unsecure environments.

Then we gave a glimpse few on um what we introduced uh last shortly. Um the testing framework um Viper and of course we uh give an overview of all the use cases we currently tried out with uh open duty what currently works and um after that we talk about our community and we give an outlook of what's coming next. But before we really come to start, we

want to introduce ourselves. My name is Mika. I'm working in Mercedes-Benz tech innovation for about 20 years right now. Started as a requirements engineer. And uh when the MBTI become more agile, I changed my profession to the product owner and had uh several different projects till now as a product owner. And in uh 2025 I had a chance to to join the first open source project of

my life. It is open. And um so from besides the work um I have four um hobbies. Uh they are called Yonas Leah and Alina and Amelia. Um and there's one more. I'm a volunteer firefighter besides that. So uh that's from my side that you know me and uh Pascal do you want to introduce yourself? >> Sure. Thank you. Oh, I'm very loud. I have to be

careful. Yeah, that's me. I just realized that's the same outfit somehow. I have more than two one outfit. I guarantee. So, I'm um Pascal. I'm working at Mercedes-Benz research and yeah, I studied a long time ago software engineering in Germany. Then I did a PhD in the same area and then after some posttock experiences and a lot of open source activities and so on I ended up

at Mercedes and there I'm basically responsible for managing STV projects internally and also externally so externally for example I'm very much involved in Hi STV some of you might know that but we also have some other projects going on and yeah that's basically my job sometimes I can do get some coding in but mostly It's STV coordination right now. So, okay, Michael, please continue. >> Yeah, let's

go into the pain points. Yeah. Um, the challenges we meet. Um, the first thing you already heard uh last days. So, the complexity of uh creating ECUs, more than 100 ECUs in a modern sedan and luxury sedan and 150 million lines of code. So the complexity itself it's it's a big thing of course but upon that um these ECUs are not singular things they have to work

closely together. So uh integration is a very very must have in all the ECV development. So but on top of that the ECUs are developed not in one single place. So normally the hardware development and software development are located uh somewhere else and the all the ECU developments mostly and if you want to test them you have to bring the tester and the ECUs together. So either

you you shipped your ECUs or you fly the E testers to the ECUs and as you all know testing is not a oneshot or one thing. It's a kind of a logistic nightmare to do some testing. So continuous testing I guess not. Um so you have this all this lag of hardware delivery delays all the time and um when it comes down then to integration you have

to collect all these ECUs building your test benches your your hubs and these um test benches to run and the hills to run is a very costly and it's very very uh cost inensit incent cost incentive and and timeconsuming uh part of That's why uh where we start normally quite late with that and with um um I would say uh most mature uh developed ECUs because the

ECUs is more the from from the from the pro uh from the process perspective the remodel does not help you as well. So normally you develop your ECU in a kind of a own project a singular uh like a silo to some kind of maturement and then when you developed it you bring this together and then you recognize oops it doesn't fit very well together to the

other issues because you've overseen something that things have changed the communication issues and so on. So all these together come up with that that you have uh very late a lot of findings you have to tackle and you have to test it and you have to restructure refactoring a lot then you have to test it again. What pushes the test effort in the last phase is very

very high and uh what makes a lot of what what drives the effort for testing and and the consumption of the very expensive hills and test benches very much. So as you can imagine it's not real good dream setup as you know this is why there are a lot of um initiatives how to tackle this there are different um ways to work with one big thing is

virtualization of the ECUs that are that you are possible to test the software even without any kind of hardware and this is working and this is a good thing especially when it comes up to scaling of course but here again if you have virtual ECUs there are some parts which are not really fulfilled for example there are dearations in the hardware development do you it's hard to

test the real real time uh behavior and the real behavior of the hardware itself when it comes together and if you do it that late you cannot uh then you have to deal it again late so here you have to tackle that some way yeah and of course You can have interface problems as well when you test the virtual ECUs because you're testing virtual ECUs not against

hardware. You test it against simulations and so you have to ensure that the simulations reflect the hardware mostly the case sometimes not. So there's the chance to get some other uh mistakes. In addition to the virtual testing, we have this testing farms always already in place and this really good thing because that makes you the ECUs available uh for a longer distance. So you can make remote

ECU tests of course but here you have again this log uh transportation thing. Yeah, you have to collect all the ECUs you want to test in one location and you have to run them and so on. So in general here didn't change too much except one thing. you have a remote access. But here you have also the problem when you run this uh in a local place

you have to run it of course and you have to take care and normally these uh setups are quite stable. So if you have a ECU for a component test, it stays a component test. You do not reuse the ECU for another integration test. And if you want to make an integration test, you set up test benches or hills and they stay as they are. So if

you need another kind of topology, another setup to be tested, you have to introduce the test bench or you have to rewire and this is also something you have to manage locally with persons who doing that is rewiring with a lot of effort. So talking about that what is the um the add-on open delivers to you? What what is this what open do enables and this is

you can come from a virtual testing to a hardware based testing easily without big changes in your testing setup as a developer no matter where the location is. So you have it pos uh you have your ECU available. It's virtual and then you can just deploy it and and test it and if the hardware comes in you don't care about the location because you have your remote

access and you have the possibility to make a dynamic uh um um integration test that you just have access to a couple of ECUs which are combined to one big network and of course with multiple locations. In addition to that you can combine them. You can combine the the hardware ECU set uh setup and the virtual setup that you complete the setup we want to test with

virtual and hardware things. And this is the way it could go to enable your testing as early as you can think of as early as it's possible. Virtualize without any hardware after first hardware comes in. put plugging in some kind of hardware in your test setup and extend it. So we heard a lot of the pitfalls or a lot of the challenges uh a lot of u

the the pains of DC development and now it's coming up what is really open do and this is uh so the main components about the open dot when you think about open do you can think about like are you using just uh um the the wiring the the the vehicle wiring harness and cutting through the cable and plugging the internet in between. So that is no matter

where the ECU is located but the the tests remain the same. The ECU doesn't care about even don't know that open D is included. So the test is quite quite the same and how to reach that we have here two user interfaces graphical user interface to click everything together if you want but of course we all want automization. So we have Cleo as a command line interface.

This is what you need for automatization and this is what we do normally in the ECU development. Both interfaces talk to KL and KL is the back end where you um just uh include and manage your atgars. Edgars are our clients or peers and you manage your clusters and the cluster is where the private network we use for the communication of the uh the clients of of

the ECUs itself. I would say yeah the client the peers here in the network we call it Edgar uh they are used to plug in the real hardware devices yeah so this can be ECUs that can be sensor there could be additional software you use for example to trigger tests your test framework whatever you think what needs access to the bus you want to have do want

to test you you need for your tests currently we support automotive Ethernet and can maybe in future more there something imaginable we come to that later again and now we want to go more into detail and how it really works about architecture and so on and for that I want to hand over to Pascal thank you very much so thank you very much for the great motivation

so I hope everyone now knows why we need like this new approach for distributed testing why it makes sense I just want to emphasize once again so if you think about what is currently done in hill testing Here in the V model, the integration testing where we bring together all the ECUs of the different suppliers is like one of the last steps that you do. And if

we find a mistake, an error there, the costs are immense. And the next thing that we have is that our hill clusters are a huge bottleneck. So they are basically booked up to the rim. So we have like kill clusters booked like day and night to run tests and you all do that in the end and you get problems with delays in SOPs if you find errors

very very late. So once again to emphasize that this solution is not something like nice to have. It has like real potential to save money to be more efficient and I would also say why are we doing that open source it can also have a benefit for other companies. I will come to that um later on not just for us as OEMs but also for suppliers and

also for all different domains. I will come back to that a bit later. But now let's dive into our architecture a little bit. So Michael already introduced like the main components that we have like a central component, graphical user interface and so on. And that's basically what you will get when you check out the GitHub repository. Now you will get access to all these components. The great

thing is we use Docker. It's very easy to set up. So I would say one command you have such an interface testable on your machine. So now it gets a bit better bigger. So don't worry I will explain that step by step. We have here um like an exemplary scenario build up where we want to interconnect different ECUs to test them together. Just imagine we want to

build up like an ECU test where we have an ECU connected to a rain sensor of a vehicle and another ECU connected to the wiper plates and we want to test them together to see if the rain sensor detects rain the wiper plates of the car turn on. So that could be a scenario and now we could say okay what do we need for that? We have

our ECUs we have some rest bus simulation. So you are all from automotive I suppose but west simulation gives you the means in testing to simulate signals because we don't have actual rain in the lab right makes sense so and we want to interconnect these two ECUs so let's imagine this ECU is produced by a supplier that is located in Berlin and this ECU is um provided

by a supplier in Stuttgot and we want to test these two together while they are still at the supplier being developed. So what does the supplier have to do? He needs edge devices which we called on the previous slide peers. What could be such an edge device? So usually what we do is we use like very cheap devices for that because we don't want to introduce some

additional costs. What you can do is for the edge devices you could use Raspberry Pies or other microcontrollers whatever you have available and I think every company has some batch of Raspberry Pies already available. Why do we need this edge device? So if you're working with ECUs you cannot connect an ECU directly to the internet. That's not working. They're working on like CAN signals automotive Ethernet lin

flexay and so on. So we have to somehow first connect the ECU to the edge device that is then responsible to take the signals and bring them into the network. So that's very important. So the edge device usually uh we use Raspberry Pies. One um I wouldn't say limitation but one uh characteristics of open dot is it runs on Linux only. So this edge device has to

have some Linux distribution. on this edge device you can um run the ATK client. So each like supplier will basically set up the ATK client on their Raspberry Pi. You can configure it by telling it basically which network interface to access to get to the signals. Now you might wonder how do we achieve the way from the ECU to the ATCAR or to the edge device. So

here you could basically use media converters or other adapters that are available for CAN automotive Ethernet that are very cheap to buy and then can basically um plug into the Raspberry Pi in the Ethernet interface. Then you can configure at Galike that that it basically consumes the signals from the Ethernet bus and is then able to send it out in the network. So let's assume all the

suppliers do that for the ECUs. So now we as an OEM or whoever wants to run test has the flexibility to simply select the devices they want to access and test together and build up their hill cluster on the go and that's very nice because building up hill clusters is very expensive and takes a lot of time. So okay so that's done the setup as you can

see we use here uh the open source library network which is a library that sets up for us the connection to the outside world so it's an networking communication opensource library um of course the communication between different suppliers and also between suppliers and OEMs needs to be secure right you don't want to lose signals it's easy use under development we don't want want uh to to have

any signals get out in the wild. So what we're doing is we're using wire guard as a VPN solution. Okay. So every supplier sets that up. That's done. Okay. Then there is one central component in the cloud, our call component which is basically responsible for managing the whole network. So first step setting up your edge devices and then you're going to the call component either via the

graphical user interface Leo or via the um command line interface Leo. Now if you want to have like automated CI/CD CI/CT pipelines then you might want to do like the setting up of the test fully automatically. So you do that you go here you say hey I want to register a new ECU this one for example. So what you're doing is you have to enter some properties,

some characteristics of this ECU, which network interfaces it wants to use and so on. And what you get back after this registration is done is a setup string. So you get like a string with some different parameters which is encoded in some format. So it's a large string. This string can be used to start the ATK client down here. So you start basically the ATKA client and

give it as a parameter this large string that you got from the central component. Once you did that, this ATK client first sends out a ping to the car component to say hello, I'm available. I'm now part of the network and can be used for testing. So once this is done, I can cluster multiple of these devices here together in the car client over the graphic user

interface and say okay these devices are now part of my virtual hill cluster and once I did that I basically can interconnect them together and I can run my tests. So that's very neat. So those are basically the steps that you have to take and the important thing is that the communication runs between the different edge devices here. So it's like this communication path here because we

want to set up a peer-to-peer network as it would be in the hill. So we don't want to route all the traffic over this central component. So it's not like a centralized communication. It's peer-to-peer. Sometimes however due to it we we all know how that is there's some firewall that prevents us to have some direct communication altogether. So in this case what we will have to do

is we have to reroute the traffic using this relay server over here which is like a solution that is called a turn server where you can still communicate using the central component between different components that could not communicate because there's a firewall in between. You might know this technology also from team viewer or from different um like tools to interconnect different computers that have firewalls in between.

Okay. So that's the architecture. That's something you have to do. So if you're a supplier, you're basically doing this part and register everything here. If you're a tester, you want to set up your virtual hill. You do that also in the graphical user interface or over the command line tool. most of these uh things I already mentioned but um this still also gives like a nice overview.

So we have like um the user who is mostly using uh the graphical user interface or the command line interface to register new devices to cluster them together to a virtual hill or we have CI/CD pipelines that do that for you. For example, it could happen that overnight you want to write an uh you want to run an integration test considering multiple devices and then you want

to run another test then another test and so on and then you could basically dynamically cluster device together test them release the cluster and cluster and create other cluster configurations. Okay. So now we talked a lot about testing what we also provided is Viper. So it's there in the first version. It's not like very mature yet. I would not say that it's a fullyfledged automotive like testing

tool yet. However, it provides you basic means to run some simpler tests based on like the cluster that you created. So um Viper is a testing runtime where you can have some simple assertion tests based on the signals that appear on the network and Viper could basically run um in the car component in the central component to run tests or it could also run on the different

edge devices and we can connect everything also to a restful simulation. Um we also did some experiments because I'm talking about rest simulation a lot. We do not provide a resp simulation as part of open dot. However, you can easily integrate your resp simulation. For example, I think most of you know vector canoe. That's like a basic tool to use for respos simulation. You can easily also

integrate that into our network to simulate signals that you might need. Yes. So, okay. So, we of course tried that out. Now, we I'm not just telling you it works. We also tried it out. We have um some different suppliers that we're working with where we were able to run some tests and see if it really works. So the first uh thing that we did was uh

we had a route from tom. So our MBTI colleagues are in M. So that was very practical. And we have some test farms of course here in in Stoodkat um where like the Mercedes Hog company is and we connected some two ECUs and we tried to figure out if it works and we saw ah it's a stable connection. We had like 18 milliseconds latency as average. Of

course latency is always an issue. You have to think about which tests can I run which tests are very like uh real time critical there. We have to think about can I test an airpack control over open dot difficult now if I need a response time of like under 5 milliseconds with the latency that I have over the internet difficult we can discuss that in the aftermath

it's an interesting topic what else did we do we also experimented with uh flashing flashing of course is something important you know because you have like ECUs those are our hardware boxes however I want to test like different like software versions And what would be cool is if I could remotely flash different software versions on it, create my cluster, test everything, and then flash another version on

it. So that's nice. And we also tried that out with solution, and it also worked over the open network, which is great. What else did we do? Security testing. So security tests, they are always very nice because they usually Whoops. they usually they don't have like these hard realtime requirements. So we ran some uh pen tests, some complete pen tests, some shorter ones, pen tests on higher

distances and they all basically had reliable results which means they worked as they would have worked in like our hill clusters test farms. And one cool thing that we also did is to set up a hybrid setup with one of our suppliers because you know that nowadays a lot of suppliers and also OEMs are interested in providing their ECUs as a virtual ECU, right? Because um it's

more flexible. You can um like have it much more early in the development process and of course for us it's interesting. Can we test together a virtual and a physical ECU using open DOT? So can we still interconnect them and test them together? So we had a supplier um from Iningan um who does a lot of like virtual ECU software for us and we interconnected basically their

virtual ECU with an ECU in our test farm together and we ran some easier tests and it worked all worked out. So this will be specifically important in the future to see how can we set up these hybrid setup because a lot of these ECUs are currently being developed as virtual ECUs and only in a final step as physical ECU. Okay. So we did a lot of

things. It all worked out so far. There are also some challenges. We can talk about that later on. Okay. So I already mentioned we are doing that all open source. Yeah. So um our company is not known for a long time to to be very active in open source. However, that changed over the last years, right? So um Mercedes and also the MBTI um they like have

now a lot of activities going on in the open source because we see a real benefit here. So what are our hopes for open source? Our hopes are that we find partners that can basically accelerate like projects such as open DOT. We don't want to do that only on our own and it can also bring a benefit to our suppliers to other OEMs. We want to share

that we want to have contributions from other people as well to basically make this bigger in a shorter amount of time. And of course we are also willing to um put some effort in. So all the resources that we put in over the last 3 years so it's already a kind of mature project. We want to continue that and for that we need partners. So we also

want to take of course the opportunity today to see if someone is interested just check out our project go to the official open um Eclipse site um use the chat room that we set up there you can easily write a question and we will answer it um in some short time and um you can also visit this site to see how to easily get involved into the

Eclipse STV that you all know um the results are also part of some um of the Eclipse STV as Michelle also mentioned in the beginning and um have been part of a publicly funded project in Germany which is called softy car and is also part of the project hala STV funded by the European Union. So how are we how we are using it right now? We are

using it inhouse in some projects. I cannot give you all the details but um sometimes for also maybe for some partners here in the room it could be interesting to not only use it together with other partners and suppliers but also to use it within your company um for some use cases. For example, we have like device farms. We have like device providers that have like all

different kinds of ECUs available at one location. Now I can also use open DUT and say hey I want to create a hill cluster based on this test farm with all the different ECUs available. I can register them all to our um platform. I can build a cluster on the fly dynamically and I can run some tests. So even internally it has a great use for us

and of course with suppliers together the use is even higher. Yeah. Um also we have um some activities going on. I also mentioned Halifa STV. Yeah, we are currently um evaluating um like uh different scenarios with different um other partners in the project. However, we are also trying to make ha uh to make the open do project a bit more efficient to integrate uh like things like

open 1722 to make the communication more efficient. We also had a talk at OCX this year and um yeah also to to integrate some let's say some concepts or some implementations that enable like hardware abstraction to have like easier access to like ECU functions for example covesa vsss or similar. So there we are trying a lot of stuff out and we are also very motivated to have

open as part of score. So there are still discussions going on I guess. So I'm nearly at the end. So I hope uh everyone is now fully motivated to contribute to our open do project. So and we are still working on it. So we are not done and we put it in a drawer. No it's still going on which is nice. So uh we are currently working

on uh a hosted service. So you can basically have a minimal set of features of open dot which can you can use like in smaller companies or in smaller teams of your company to have like a quick setup of everything that you saw here and can immediately start trying it out. Then um we are currently working on Viper some more. So I mentioned Viper like the um

test execution engine. It's currently um like not that comprehensive. So we are still working on it and we are also working on integrating Viper better with open DOT and of course we want to make open D do D do D do D do D do D do D do D do D do D do DOT or open dot ready for broad usage and for broad usage we

have to think about also about some uh certifiability maybe some compliance things maybe so something that we have to think about how can we really bring that out in the wild because right now it's like an open source project we developed it for 2 3 years but still there's a lot to do to make that a bit more robust to make it more um more secure maybe.

So some things still need to be done and then we're thinking about how can we uh add new features restful could be that we integrate a very light b lightweight rest simulation into that that has the possibility to simulate some basic signals could be however many partners will be using their own rest simulation and testing let's see and um also we have to think about how can

we like integrate further vehicle bus systems Um that's also a decision that I would say we have to think about because um we now have CAN and we have automotive Ethernet which are like I would say the ones that are mostly used. We have to think about do we want to take the way to also support like something like flexay or is will this be part of

the future? I'm not so sure. So I still have some question marks about that. So I think we are pretty mature so far but it would of course be great that some of you tried it out and think about how could you help us how could you support us in bringing open boot to the next level with your contributions that would be very important for us. now

like some animations. So open D is like as you know part of the Eclipse um STV. So you can immediately try it out easily. Set it up connect your devices to enable like better ECU software development and to make it more efficient and much faster. I can think of a lot of use cases and in the beginning I mentioned that this could also be interesting for other

domains. It's not just like the vehicle domain or the automotive domain. I can also think about use cases for like um aircraft for example. So basically all domains that have easy use involved and that's a lot of domains that could use that and need to have a more efficient way of testing. Um yeah, thank you from my side. Michael, some final words. You have some some remarks

left. [laughter] >> You are everything there. >> Yeah. >> Thank [snorts] you all for your attention. We are >> Thank you. >> happy to have your questions. Of course. >> Uh thank you Michael and uh Pashkal for the great session. >> Um I have one question before I hand over to the audience. Have you been thinking about device abstractions for the for OpenD? I know that we

are dealing with physical devices and device abstraction is a uh is a stretch mental model but how do you deal and how do you think about these for the future of your platform where you want to orchestrate the platform and relying on the device to tell you what they are instead of you chasing the device for what it is. >> Yeah. So that's a great question. Um

so it's something that I think about a lot because right now we have like um this trend going to more like device hardware abstraction to not only work on like this very lowle um signals and be very hardware dependent I want to be more hardware agnostic right and I'm I'm thinking about in the scope of the hala STV project at which level can I integrate like a

layer of hardware abstraction to write my tests basically not based on the low-level signals But based on this hardware abstraction and what would be a good way to do that right could we somehow incorporate at some layer VSSs how's the communication then being done because what I also don't want to do is I want to don't want to introduce a central component so we have we have

to think about how does that affect the architecture here and I would say right now it's just ideas in in my head but we are definitely thinking about that intensively yeah >> because Um yeah that's great and again it's just a shameless plug. um my previous life [snorts] uh at Microsoft on the SDV working group we did an initial project on device abstractions for the ECUs we

call it IG at that time it's a one of the open source projects on STV and I'm always thinking about that right how we do that because in the end dealing with physical devices shipping them like you you are just solving the right problem the next problem is definitely that device abstractions and how we abstract from the low-level stuff uh into something that the developers and the

tooling um and to with AI we can think just about um how AI can help us uh set up all these again thank you so much I really appreciate your session >> thank you great >> um from the audience questions hi so I'm just trying to understand generally what this exactly does uh every time you say you're testing or you're running a test I guess what you

mean is is a black black box kind of testing. So stimulating the ECU with external signals. This doesn't encompass let's say loading software into the ECU, configuring it for let's say more uh particular tests like unit tests or things like that. Um yeah, that's actually not the scope of this project. That's true. So we are currently um talking about integration tests. So basically we have like these

different ECUs that we treat as black boxes. So usually we get that from suppliers. Usually you don't even see the code. So if you have like the the box with the version on you can basically commission another version software version to flash on the ECU but you have that only as a binary and we are like focusing on how can we see if the ECUs work together

properly properly. So we are not looking into the boxes. We are looking at the >> Okay. Thank you. >> Thank you for clarifying. You're >> welcome. Yeah. Any more questions? >> Hi. Um uh great talk by the way. Um thanks. My question is uh how do you handle synchronization when it comes to the send response on CAM messages for particular test scenarios? >> Yeah. Do you want

to >> synchronization? >> Synchronization. Yeah. >> Um this is uh definitely still a big topic. So um there are some some works in uh synchronization. Um got to say is nothing implemented in that currently right now. It's just having the peers and uh so uh a lot of tried there are some some ideas of having some kind of header protocols synchronize the the the the signals that

you can also rely on synchronized uh aspects for for the tests. But this is um yeah this is a a thing we have to consider and currently um I guess two of our partners there which uh which came in in the last 3 months are very interested in that and want to contribute exactly in that point. So this is the very beginning and thinking about synchronizing the

the messages. >> So still some stuff to do but that's a very that's something we think about a lot as well. If you go over the internet then you have these issues all of a sudden that you wouldn't have in your hill cluster if you use a cable. Yeah, that's exactly something we think about intensively here but not yet done unfortunately. Any other question so if that

there is no other question again thank you so much Pascal and Michael for the session. Um we are on these world of online meetings. So I will give you back 4 minutes but it's not an online meeting. So enjoy the networking and join the lunch. And once again thank you for the great session.