Benjamin Nickolls, Andrew Nesbitt – Is InnerSource Commons good for open source? #FOSSBack

This talk explores the impact of Inner Source Commons on open source contributions using data from their project called Ecosystems, which tracks approximately 14 million packages and 280 million public repositories. The speakers, Ben and Andrew, analyze the contributions of organizations involved in Inner Source and aim to answer whether being part of this community is beneficial for open source. They present statistical findings showing that about 10.6 percent of maintainers contribute to open source projects and examine how contributions are distributed across organizations, emphasizing the significant presence of Microsoft. They also discuss the challenges of measuring contributions and the importance of understanding the health of dependencies in open source projects, suggesting ways organizations can increase their contributions and direct efforts more effectively.

Hello, hello. Sorry about that. We're a little bit rusty. But we're here now. And then I have another apology to make, which is that we're all subject to a pretty bad joke right now. This is a very dated reference. I don't know whether anyone remembers kind of early 2000 Ashton Kutcher used to have a bit of a career. Yeah, cool. Um We are here because last year

I asked a question and the speaker Willem, I think you are here. Hello. You remember me? I'm sorry. I asked a question about inner source commons and the impact of inner source commons on open source because in Willem's talk, I think you were talking a little about the debunking the idea that inner source is a detriment to open source. We didn't have any data in that particular

talk. So, I decided to submit a talk to basically answer that question. And now here we all are. So, hello. I'm Ben. I'm based in the UK on the border with Wales and you are? Hi, I'm Andrew. based in the UK as well. About 10 minutes from Glastonbury Festival site, which obviously you might recognize Glastonbury Festival and then think actually I have no idea where that is,

but I live near there. Uh we both work on a project called ecosystems. This is an index and a kind of a system of tools and APIs to collect data about open source projects, about packages published to package managers, the dependency graphs and all kinds of other information that you uh want to collect from public kind of use of different kinds of open source. And we collect

a lot of it. We're tracking about 14 million different packages, about 280 million public repositories across about 2,000 different hosts, and the biggest number, and the one that causes me the most headaches, is this 25 billion dependencies. We mine the dependencies out of every public repository and then connect the dots between them to be able to say here's a graph of how everything is connected in open

source. And this enables us to do all kinds of interesting analysis and kind of go spelunking around and connecting dots that you would otherwise be like, how on earth am I going to like put these pieces together? So, as Ben has so kindly roped me into, we're going to try and answer this question of is inner source good for open source? And to start off with, uh

we're not trying to take anyone down or to make inner source look bad. We are purely using some public data to try and highlight some interesting problems and challenges that come about from companies uh trying to work with open source and contribute to it. Uh we have approximately 10 years of experience working in the open source data space, kind of spelunking around all of this stuff. We'd

actually kind of already given this talk. Uh when was this? >> Nearly 10 years ago. >> Nearly 10 years ago, but focused on the UK civic tech scene, where we went and looked into the people who produce open source as part of civic tech and see how much they contribute to the different pieces that they use as well. I would just say as well, sorry to interrupt,

but um one of the other things as well is like we appreciate the inner source commons is not all about contributing to open source. As we've just heard, there are many benefits to practicing inner source uh in terms of productivity inside of an organization, but we just wanted to kind of answer the question uh that I asked last year, so. Okay. So, first of all, how can

we define inner source in our data set of ecosystems? Uh, the website basically says that there are uh, 3,000 individuals across 800 different companies and academic institutions and governments, that we were like, "Okay, great. We'll just go get that list of 800 companies and start there. That'll be That'll be easy." Uh, but not so much. Uh, we went going to look for that and there isn't actually

a a list of those companies. So, instead, we scraped it off of the website. Um, we took what is kind of like publicly available as companies saying that we are involved and then mapped that back and worked out which GitHub repositories each one of those has. Potentially, some of them have multiple different GitHub organizations, uh, that kind of all live under the umbrella of that one company.

And then we went scraping of my own API. Uh, so all of this is based on public data. I didn't cheat. I didn't go and do SQL queries on the database, but instead used the API and wrote scripts and there'll be links at the end if you want to go and and spelunk into this. Uh, this is not going to be visible at all, but uh, the

resulting scraping produced nearly 9 GB of JSON across 350,000 JSON files, basically one for every different API request that I made, and then I compressed that down into a SQLite database of of nearly 500 MB of different bits of data related to all the things I needed to be able to collect to answer the questions that Ben had posed, uh, not that long ago. >> I I

I you in it. I'm sorry. This is how we work. So, from our starting point of about 108 organizations, which I was able to get from the website, I then went through and looked at all of their public activity across GitHub. Almost every one of them is on GitHub. Which then I looked up for each repository, which repositories Sorry, which people look like they have some kind

of commit or maintainer status on those projects. So, we're not just broadly going anyone who's ever touched these projects, but the ones who actually look like they have some kind commit bit or maintainer bit. And there's a number of different signals there that we don't have time to go into. We then noticed that 50% or so of a lot of this data makes up becomes from one

gigantic organization, which is Microsoft. For one thing, they have at least 13 different orgs for various kinds of projects, which all kind of sit under the same umbrella. We're also not including GitHub as GitHub the org as part of Microsoft because it kind of stands alone and has a a bit of a history beforehand, but that's also, you know, it'll show up that there's another big group

there. And in terms of actual activity, this makes up like 50% of all of these organizations' amount of activity. So, we then have to go and make another SQLite database without the Microsoft data in there just to be able to say this is massive and will absolutely skew our data. So, can we Can we kind of like pull that back and look at what it looks like

if you take away Microsoft? having defined the sample set, so we've got our 108 organizations that are members of IC, or at least publicly, and the GitHub repositories and the maintainers within that, And next thing we might want to define is well what does good look like? Um and that's quite a complex question and one that we could spend the whole time kind of talking about but

I think what we're going to do is just focus on some specific questions and then we will probably together kind of get towards a answer to that question that maybe we can agree on. We'll see. Um so the first question that we wanted to answer was do ISC members publish well used open source software? And looking at the data set it looks like we have around 3,776

repos publishing around 14,000 packages within that data set and of those packages we see around 8.25 million dependent repositories. So these are Git repos that depend upon any one of those 15,000 odd packages. And amongst all of those packages we see around about 20 29 million downloads. So not nothing. I think it's fair to say that ISC members are publishing some software that is well used. Um

as again we're not going to continue saying this but Microsoft does dominate the picture. This is a breakdown of dependent repos by ISC published organizations and you can see that Microsoft is in there with TypeScript and with TS Lib and with Rush Stack as well. And then after that you have GitHub who's got a couple of Jekyll gems that are incredibly popular as well. So massive data

skew but we're going to stop saying this at some point. The next question and the question that we really wanted to kind of get to is do Inner Source Commons members contribute to open source? And actually we see some really good data there. We see that 10.6 percent of all of those maintainers that were identified do actually contribute to open source projects. They've contributed about 4.6 commits

I believe that is. It's a combination of commits, issues and pull requests to be specific. Oh, nice. I'm glad you're here. not just a pretty face. To approximately 100,000 open source repos. And again, we've got a breakdown of that here, so many contributions by I see. You can guess what the big block is. Uh I won't go into that anymore. Um and then we wanted to ask,

do members contribute to their dependencies? So, the talk that I did nearly 10 years ago at a conference called Tic Tech was in the wake of a few things that had happened, but ultimately the civic tech community was kind of looking down navel gazing and wondering what it was actually doing. And the message that we want to give is what does civic tech as a community contribute

to the more kind of global kind of open source commons? And we get to something like this um here as well, which is what The question we're trying to answer is to what degree do do ISC members support their own foundations? And actually of the 2.6 million dependencies shared by the 108 organizations that we found GitHub organizations and repos for and the 51,000 unique packages that were

shared amongst those um those organizations 59, so over 50% of those organizations did contribute back. It should be mentioned that we're only looking at the public repositories of these organizations as well. We don't have access to their private things. We're just inferring how much stuff it looks like they're contributing to based on what they use in their own open source >> Yeah, that's right. So, um around

3,000 4% of the people that we saw associated publicly with any one of those 108 organizations contributed back. Um they contributed to about 10,000 of those packages, so roughly 20% of those um have been contributed >> Guess how many came from Microsoft. Many. Um so, I I'm going to pass back to you to ans- answer the question again. Is that good? Well the good is a is

a question there, but let me first uh kind of introduce you to the concept of open source being free as in puppies and I by law have to include pictures of my dogs in every single talk that I give. Here are five poodles um that basically take over all of my life. Free as in puppies works really well because you have uh lots of kind of concepts

of uh packages and open source projects that people are depending upon. You can have it for free, but actually it comes along with a load of problems. If anyone has ever looked after a dog, they're like, "Oh, actually like this is quite a lot of work." Uh if you looked after five dogs, then you're like, "Oh my god, what have I done? This is a terrible idea."

Um Also, uh kind of stepping into this further as uh talking to Ava this morning was like, "Oh, this is this is a really interesting angle here that you haven't uh thought about." And this is something that I don't think anyone has really mentioned yet today is the CRA. The Canine Responsibility Act uh and so I consulted with my um law firm, Nesbitt Nesbitt and Nesbitt. That

is Basil, Mabel, and Felix, attorneys at paw. What is He didn't know I was going to do this. Contributing to uh your dependencies isn't just uh kind of being able to be a good citizen, but actually this becomes part of your kind of requirements of compliance as part of the CRA. Companies that are building relationships with their upstream maintainers are going to be better positioned to follow

through on their CRA obligations when they kick in. If you're selling a product that includes a puppy, then you're going to be responsible for that puppy's health. This is going to get weird, but it's okay. You need to know which puppies you have as in your S-bomb, you need to know whether those puppies have had their vaccinations, the vulnerability management, and you need to know who the

vets of those puppies are, as in the maintainers. You see, I can really draw out a good analogy there. If the puppy bites someone, that puppy is now your responsibility. Uh might not want to take legal advice from this particular, uh organization. Um But, we can I We've color-coded these things as well, so you can tell. Like, green means good, right? Uh When we went and looked

at the contributions that these, um organizations had made, broken down by issues and pull requests and commits, you actually see that one thing that I was worried about is that there would be lots of people opening issues and saying, like, "I need you to fix some things for me." And they're not actually going and following through and making contributions. But, actually, nearly 90% of all the people

that we looked at who had opened issues had also made some open source contributions. So, excellent amount of follow-through on that. They're not just reporting problems, but actually trying to fix them, too. the breakdown on this is obviously massive, because you can have, um many more commits in a particular pull request, and that people are often, when they're a maintainer, they're not necessarily opening pull requests for

everything. They're just committing to main and just just launching it out into the world. But, uh these people are actually doing a lot of heavy lifting in a lot of these open source projects. Again, you can imagine there's there's a large Microsoft contingent in there. And uh four-ish percent of all of those commits actually went to the dependencies of the ISC projects, which is Again, that's a

really significant number, if you think about it, the amount of open source projects that they're using and all of the contributions they're doing they're actually like able to say like we're having a noticeable impact on the projects that we use. Uh comparing that back to um Ben's talk 10 years ago, we see approximately 30 commits per uh ISC um member. The um civic tech movement had uh

nearly 130 commits per person, but civic tech is you know everything is open source basically. So there's a quite a different cultural uh leap there as well. Um although the the approximate numbers of the two different groups we were looking at are pretty similar in around 3,000 members that we were inspecting. There's also this kind of term of critical projects and this is more broadly like packages

and pieces of software that are critical to everyone. They are they make up like 80% of all of open source usage and that's maybe total 10,000-ish projects, I think. And actually when we look at the contributions to those projects, we see of all the contributions anyone has made, critical projects 1.2% came from ISC members. Again, there's a Microsoft asterisk in there somewhere, but that's still a a

significant amount of contribution that they have made to projects which are used by basically everyone across the Okay, so once again, uh we want to ask this question, is that good? Right? So is the amount of impact that we've seen is the fact that we have a bunch of maintainers, 3,000 odd, uh contributing to open source in a way that is not I would say extractive. We've

got 90% of people who raise an issue contributing back and we've got 4% of all of the code that is written that supports the foundations of those members and 1% of all kind of critical open source contributed by OSC members or at least OSC members that we could find publicly. could we say that that is good? Uh uh I think you definitely could. You definitely could say

that was good, but I think that we can do better. and the question is how? Uh and uh you wanted to stretch uh a metaphor. I'm not stretching a metaphor. I'm just showing a photo of a very good-looking man. Um because this is Donald Rumsfeld. I know he used to look like that. Jesus. who came up with the known knowns, known unknowns, and unknown unknowns. And one

of the things that we often hear from organizations is that they find it very, very difficult to work out what software they actually depend upon across their entire software estate, let alone working out what a group of companies depend upon collectively. But, we built a platform exactly for that, so we can solve that problem. Just hovering over that slide there as well for as long as possible.

I'll keep it up there for as long as I want. Thank you very much. Um so, looking at that data set, we can see that around 9,300 of those dependencies, of which remember there was 51,000 of them, so we're already getting down into 20% rate, are depended upon by more than five member organizations. And then we can chart uh this is a chart of uh commits and

issues, commits in green, issues in red, against the number of organizations that we've uh found to be ISC members are using. So, we have left the most critical to as many of those members as possible to right most to least critical. And we can see the activity of those members within each of those repositories. Um so, you can see here the graph is kind of skewed. It's

kind of skewed low. So, saying that we have a good group of people who are contributing to open source, I think collectively that group of people could combine their efforts to have more of a positive impact for the entirety of that membership. And there is a long, long tail that goes out at the end of that graph. >> We We couldn't We couldn't fit it all in

the slide. Is that what you're saying? Mhm. Um and then we have uh a number of dependencies, so 4,000 dependencies, which are also critical to the entirety of open source. And again, we can graph that thing. And we can see that again, it's kind of skewed low. Yes, this is only the the front kind of part of the the tail, but we can see again you could

probably focus those efforts a little bit better. if we go down and take a look at where potentially those projects might need a little assistance, we get into a different conversation. Um that conversation starts by us identifying the 4,000 of those projects appear to only be maintained by a single maintainer. And then trigger warning for anyone at home. Uh 3,377 of those uh dependencies do not appear

to be healthy, meaning they show signs of like no release in the past 2 years while having open issues. 1,500 of them combined kind of stale releases, have a lot of opens uh issues, or are maintained by one or two maintainers. Um health in open source is a completely different talk. Um we're not going to do that today. Maybe we'll do that next time. But we have

built a free tool for people to gain a better understanding of uh their open source kind of uh dependency uh state, as it were. Um and to get a better understanding of where each of those projects are, so they could direct their efforts. Um we also haven't spoken about funding at all, uh which is another way in which these organizations could engage in their dependencies. That is

a talk that we did uh at FOSDEM last year. You can go and watch that talk if you want, but the TLDR is that funding is misdirected. We don't have enough data to to say whether or not we are doing enough in the space. Um we need projects to tell us what their needs are in order to do a better job at that. And that um we

think we can statistically prove that you don't need to get the dependency data from all of these organizations in order to direct the efforts to the open source projects that would most benefit those organizations because we see usage and usage in proprietary software we see a very strong correlation when we look at dependent repos and downloads. So there's a whole other talk which you can go and

see on that subject. >> I can I just repeat that a little bit more cuz I think that's a really interesting idea is that we only use public data to be able to pull some of these numbers. Obviously you can say oh that's not representative of what's happening internally. But when we look at all of open source and the number of downloads and the number of public

open source projects that depend on those packages that have been downloaded. There's a very strong correlation. And what that means is for one thing there are some package registries that don't have download numbers. We can use the dependent number instead as a fill-in. Now it's not going to give you an an exact scalar of like oh yeah this is this is like this exact download number equivalent

but instead be able to rank to find what are the most critical projects in any ecosystem regardless of the availability of those download numbers. Places like go and Maven and Java don't provide you download numbers. So you're not going to be able to you need a fill-in for those things. But also we can use those download numbers that we do have to say this is for all

the global population of downloads private and public. If those things are heavily correlated between dependent numbers we can basically then say actually like we can use these dependent numbers for a fill-in to say these are also probably very close statistically to what's happening in usage within closed source as well. So I think this number could potentially enable us to kind of go you don't have to tell

us your dependencies but we can use these large numbers to get us basically to the point of telling you probably what your like statistically you're very dependent on these things based on the ecosystems and the languages that you use. Yes, perfect timing. I mean I think we're done. Um just finally to say uh if you uh interested in the data set, we've published it. If you're interested

in the ecosystems, then you can find us down there. Yeah, all the code uh that generated the data and the data is in that GitHub repository as well as some other questions and rabbit holes. >> So, I went down before we came. Cut it all back down to fit in half an hour. Do we have time for questions? You you you were talking about the InnerSource Commons

members, and you probably said what exactly you consider a member, but could you just say it again? So, I just looked at the InnerSource Commons website, which has a list of about a hundred uh of the organizations by company name out of the declared 800. Uh so, I just took those names and then turned them into which organizations do they have on GitHub? Uh where some of

those don't have any open source activity at all, but some of them like most of them that we looked at do in some form. I did not go and search every single user profile to say what company they exist in because that was way too much privacy invasion than I was comfortable doing. Okay, so just what was listed on the website as like is active in Yeah,

if an organization published a story on the InnerSource Commons website. Yeah, you're in it. Yeah. To answer the question of if is InnerSource Commons good, did you think about looking at some organizations that aren't in InnerSource to get a benchmark to compare the InnerSource people to? >> Yeah, Ben dropped that one on me on Friday night, and I was like I I can't I can statistically pull

out some other random organizations, but without looking at a lot of them, I would feel pretty bad. Uh I generated 86 GB of JSON just for 100 organizations. So, I was like, oh, we'll we'll have to save that for another time. But yeah, that's definitely a way of kind of doing a statistical comparison of like orgs that are in versus orgs that are out. But also, how

do I know that they're not one of the 700 other members of the of that the group of ISC organizations? It wasn't Friday night. Um but yeah, the So, one of the things that I haven't drawn a line to you is also that we did this talk and we could give you the comparative figure for Civic Tech. But actually, like growth in open source has come on

over the course of 10 years. GitHub publishes data on that. Like, we've seen 3x growth in contributions to open source over the course of the last 10 years. So, you know, it's very difficult to compare the two. Just if anyone needs another palate cleanser. You're not going to show Donald Rumsfeld again. Oh yeah, exactly. That's what All right. Any other final questions for these gentlemen? Going once,

going twice. Um thank you for the talk. This was really funny. I have I have no question. That's all we're here for.