About this talk
In this talk, Tim Ward, the CTO of Kentyou, discusses the challenges of maintaining an open source project, particularly focusing on Eclipse SensiNact, an IoT gateway and digital twin. He explains the project's origins, initiated by CEA, and how it aims to unify data from various IoT devices. Ward emphasizes the necessity of continuous investment in community engagement and providing adequate documentation and examples to attract external contributors. He shares Kentyou's collaborations to elevate SensiNact's visibility and utility, outlining the tools and strategies implemented, such as utilizing GitHub for collaboration and enhancing CI workflows. The speaker highlights the importance of responsiveness and inclusivity in fostering a healthy open source community and conveys key lessons learned from their experiences.
Full transcript
[music] >> Hello everyone. So, I'm Tim Ward. I'm the CTO at a company called Kentyou and I've been working with open source for a couple of decades at this point. And uh this talk will be a little bit different because what everyone wants to tell you is how to do successful open source and I've got a slightly different take on that, which is I want to talk
to you a bit about what to do when things maybe aren't going so well because as we've heard, it's not easy to make a successful open source project. So, I'm going to start to talk initially about Eclipse sensi act and what it is because I'm guessing most of you probably haven't heard of it before. Uh so, it's part of the Eclipse technology project like many many other
projects. And what it does at its heart is provide an IoT gateway and digital twin. So, it's there for uh exposing data to people, representing a city or a road network or a factory or or whatever, but it has data that it contains in its own virtual representation. It is lightweight, pluggable and interoperable. And it's this integration point for proprietary devices, different specifications, different protocols and it
can convert between them. So, you end up with this one place where you can put data in and you can get data out and it doesn't really matter what form the data was when it went in or what form you're expecting it when it comes out, it can do the work in between. And I mean really do the work in between. So, if you have ever been
in a situation where you've had data coming from a mixture of different APIs, so say you've got some Laura One devices and some web of things devices and some sensor things devices and you want to be able to expose them all from a single place, sensi act is the tool for you. It's bringing data in from wherever and then you can re-expose it with potentially the same
API or potentially a different API or integrate with multiple vendors simultaneously. If you want the the same devices exposed over multiple APIs, we can do that. So, a little bit of history about the sensi act project. And it started quite a long time ago now. Uh CEA is a French research agency for those who don't know and they did a lot of research work and they began
in the early 2010s research projects where they needed to work with IoT, smart devices and they were trying to build something that was going to allow this new world to be explored. As the project grew, they would build new features, new new parts of and it became the beginnings of a digital twin, which is what we would call it now. At that point, a digital twin was
still relatively new term. It was only really just been coined. So, they probably wouldn't have called it that then, but that's what it was. And then about 10 years ago, CEA having worked on various research projects, decided that they wanted to take the work that they'd done and put it into open source. And that was the beginning of Eclipse sensi act, which was the project that CEA
set up. Now, something that I'm sure, well, I hope you're all aware of and people have have definitely said, when you start an open source project, that is the beginning of a journey. It is not the end of the journey. If you stop investing having started the project, then the project will fail. You need to spend a lot of time and effort to build a community before
the project will be self-sustaining. You need to make sure that you publicize the work you're doing. People need to know that your project exists in order to be able to find it, to use it, to want to contribute back. You need examples and documentation so that the people who do find your project know how to use it. If you don't have those things, then you could have
the best project in the world, but nobody is going to know what to do with it. And when people ask you questions, when people offer you contributions, you need to respond back to them and you need to respond back to them in a meaningful time scale. It doesn't have to be the same hour. It doesn't even necessarily have to be the same day, but if it's not
the same week or even the same month, people are going to assume that you don't care about what they need to know or what they want to provide. Now, unfortunately for CEA, as a research organization, what you don't have is budget to uh put into doing this sort of thing. It's people's time. It's, you know, you do it in your in your spare time at the end
of the day or maybe you get a few hours allocated to do it, but you don't have a team of people whose job it is to say, "Right, let's let's make sensi act into a really great open source project." What they focused on was more research. And so, about 4 years later, we were in the situation where there was some reasonably good documentation and some examples, but
there was little to no engagement from anywhere outside CEA. No activity on the mailing list at all. Uh and really there there were no there were no external users or at least not obviously. Nobody was raising issues. Nobody was contributing anything. So, this is not a good state for an open source project, right? This is something that happens a lot, but it's not where you want to
be. So, what we needed was we needed to be able to make a change and say, "Okay, how can we take sensi act, which we believe to be great and useful, and get other people to see how great and useful it is?" And so, in 2020, Kentyou, the company that I work for, is spun out from CEA. So, this is now about 6 years ago. And the
goal was data intelligence for your city. So, it was about providing data to cities, about allowing them to make better decisions with the data that they already have, right? Collecting smart data is not a new thing for cities. They tend to have lots and lots of different projects that are producing this smart or connected data, but none of it is joined up. And sensi act was the
perfect tool for taking all of this data, joining it up and allowing the city to do more with what they already have. So, this was great for Kentyou, but one of the things we recognized straight away was we were never going to change what was happening with sensi act unless we also changed the way in which the open source project was being approached. And the first thing
we needed was we needed some people to collaborate with, right? If you've just got one company participating in an open source project, it's not really an open source community project. It's just a company that happens to be developing something in the open. So, Data in Motion, a company based in Jena in Thuringia, in Germany, uh also needed a digital twin. So, they were working with the local
government there in the city of Jena and they wanted to be able to take data that the city had and start using it to help people make decisions. So, there was a a good degree of synergy between what we wanted and what they wanted, so it was a good point to start working together on this project. So, once you have multiple partners involved, certain things become really
critical, which is you've got, if you're going to collaborate, to communicate. And you have to communicate, if you're an open source project, in the open. You can't do it on a phone call between two people that nobody else knows about. It has to be on a mailing list. It has to be somewhere public where other people can come and see how and why decisions were made and
what the results of various conversations have been. So, the tools that we used were uh certainly ones that uh for things things that you'll be familiar with. So, you have GitHub. You have tools from the Eclipse Foundation themselves. You have things like Read the Docs for being able to produce better documentation. And we started to put together the things that sensi act had never really had. It
had been, you know, code given. It was visible, but it didn't have easy ways to create new issues and contributions with the right templates and formats so people can provide the information you need to provide if you're trying to give something to open Having automated CI tasks that you can run through GitHub actions to do things like verify that there is a contribution agreement for the the
person trying to contribute something. Making sure that all the source formatting is done automatically on check-in and validated so we don't end up with a code base where everybody's fighting over tabs or spaces. Similarly, automatic management of dependency licen- licensing. You may have heard people talk about Eclipse DASH. It's a wonderful tool. I suggest you look into it if you're not using it already, but it's a
great way of knowing that what you're doing is building an open source project that's actually going to have good governance all the way through. And so, these are the kinds of tools that you get from using a good source control platform and CI platform and good tools support from a foundation like Eclipse. So, we then said, "Okay, we have these tools, but what we then need to
do is we need to enforce within the community that we actually use the issues for discussions. We use the CI actions to give us test feedback, and we need to then take the code base that as I've said is at this point nearing 15 years old. Um and we need to modernize it and make it look like a project that is living, breathing, and being used. So
then if we move to today, so what you can see here is this is an example of some work that uh Kentyou is doing using Sensnact. So this is live data that we collect about traffic in Istanbul. There are several hundred, I think about 400 traffic cameras that we're monitoring directly uh along with about 250 radar sensors and another I think 5-600 magnetic loop sensors at various
junctions. So all of this is giving us live data, actual counts of vehicles all across the city um being managed in the single digital twin. So as I said, lightweight and scalable. It needs to scale to a a large city the size of Istanbul and still just keep working, responding quickly. And then just to show you how flexible it is a completely different type of use case,
one that is in no way as fast responding, but is actually still much larger in terms of real data volumes, is uh managing urban heat islands. So you have the concept of an urban heat island, which is that the center of a city tends to be hotter than the countryside because there's lots of black surfaces, lots of things that build up and retain heat. And that's a
problem for people's health, right? If the if the physiological equivalent temperature, which is a measure of what how hot your body feels reaches a certain point, then you know, old people and children need to stay inside where it's cool. And you don't want to have that happening for weeks at a time in a city if the air temperature gets too high. So those are that's kind of
brought Sensnact to the the modern day. And then there's something happened quite recently which uh I thought was worth trying to share. So with an open source project, if you are doing well, you get lots of users, which is great. But if you're doing really well, what you start doing is you start acquiring a committer you know, or lots of committers because committers are users who are
so invested that they actually want to become part of the community, contribute back, and change the direction potentially of where the project is going. They want to become part of the community rather than just someone who is using the software. So this is the the difference between you know, how useful is your project versus how healthy is your project. A healthy project gets lots of committers. And
we got a new committer last month um and this is someone who'd never really contributed to open source before. And I actually asked them for some questions, tried to get some feedback about what the process had been like. And so this is a guy who's got uh you know, more than 10 years of experience in the industry, not with open source, but with closed source development, uh
Python, Java, OSGi, which is relevant to uh Sensnact because it's what we use as the building block for actually making a modular gateway. Um and what he came along with was that he wanted to make use of the SensorThings API to access certain kinds of data, but he'd been having problems with the existing implementation that's out there. So he wanted to know what he could do to
try and get access to data through a gateway using this API. And he chose Sensnact because it's this whole unique customizable gateway. So it's theoretically possible to add this new access layer uh to the gateway even though it's not already there. And that it's going to scale to the the level that he needs. So in terms of challenges, uh he needed to actually work out how Sensnact
worked in the first place to be able to provide some kind of contribution and work with that modular architecture. Um also he'd never really done standards implementation before, so that was a a a new thing. And then finally there was this like moving his mindset from I just do development and I commit it to having something which is more open. So getting started, hopefully there was some
prototype code for doing this already, which was why he thought it might be a good idea. Um and there was some good high-level documentation. So this getting started, getting into the beginning was was possible. Uh there were also good instructions about how to build and test what you were doing. So again, making it easy to get started. And while the CI acceptance jobs were a little bit
intimidating, actually the feedback they gave was very useful. It allowed him to tidy up a lot of his own mistakes or the areas where he wasn't doing things quite the way he needed to without having to go through a big round of feedback with people. Uh some surprises, he wasn't necessarily expecting that the review process would be entirely public. Um and he wasn't also sure how willing
we'd be to take code from a complete stranger. But you know, it it wasn't so bad. Uh so he has continued to provide more fixes um and actually since then we invited him to become a committer. And he's really happy because he sort of feels he's got that trust from the community now and the ability to to make these changes in a way that he wasn't necessarily
expecting to get on the So to summarize, I think what we can learn from this is that there are lots and lots of open source communities and in the Eclipse Foundation. Some of them vibrant and healthy, some of them less so. But how do we get people involved? Well, if you're a user, what you need to do is you need to raise an issue, you need to
fix a bug. You need to get involved. And for projects, what we need to do is we need to make that as easy as possible for these people to start. And we need to keep an eye out when someone raises a good issue, when someone provides a good contribution, we need to recognize that that's someone who could be our next So in summary, be responsive, be respectful
to the people who are coming to you and offering you contributions, and above all be kind because you never know who's going to be the next hero to provide that new piece of function that you've wanted for years. So thank you everyone. Uh if you want to talk to me more about Sensnact or anything like that, then see me at the Kentyou booth.
