About this talk
This talk focuses on social network analysis (SNA) for engineering leaders, presented by a senior engineering manager at GitHub. The speaker, with a background in sociology and computational social science, describes SNA as a method for studying social structures using graph theory. He shares his methodology for analyzing team dynamics through SNA, highlighting its effectiveness in identifying pain points like connectivity issues and silos within teams. The speaker explains how he visualizes networks using tools such as Neo4j and discusses the application of SNA to improve team performance and collaboration. He also addresses potential ethical concerns with using data analysis in managerial practices and emphasizes the importance of human context when interpreting network data.
Full transcript
Can y'all hear me by the way? Oh, I can. Okay. Cool. So, I guess I'll get started. Good morning, y'all. Um, so I'll be talking about social network analysis uh for engineering leaders. Uh a little bit about myself before I start. Uh so I'm actually a sociologist and anthropologist uh turned software engineer. Um I started out my career uh actually as a computational social scientist uh doing
social network analysis and then I spent uh the next 15 years uh getting involved in technology and software engineering space and and engineering leadership. Uh today I'm currently a senior engineering manager at GitHub. Um, but next month I'm going to leave GitHub for a local Amsterdam startup called Hydrix that's working on renewable resources. So, uh, quick overview of this talk and why it's interesting. So, uh, first
of all, I'll tell you a little bit about social network analysis and what it is. I'll talk about my methodology for exploring networks. Um I'll talk about uh some of the uh the pain points that we can surface with social network analysis and what uh potential mitigation opportunities are. And finally I'll talk about AI ethics and the limits of social network analysis. So social network analysis uh
what is it? So this is a social network analysis 101. Um social network analysis uh is a research method for studying social structures through the use of graph theory and network theory. Uh it was actually developed around the 1930s uh during the development of mass survey techniques. So think like um when when you would uh phone call people and you would ask them their opinion and these
were just really big survey techniques and also quantitative uh psychology uh where you would uh kind of test people's IQ over a mass population. So at the time uh when social network analysis was really new uh a lot of people were really excited about this uh new methodology but at the same time people were also really nervous about it because it was turning people into data and
when you start to turn people into data uh people start to feel dehumanized. So um another thing is that it evolved alongside computing technology. So uh initially all these calculations were done manually but over time they started to do the calculations uh obviously on computers and what you're looking at right now uh in the back here uh is actually a does this even work? Uh yeah it
was a uh is a Twitter conversation that happened in the US over uh a really polarizing topic. So, so why do I use this uh as a manager, right? For me, it really helps me understand my team from a completely different point of view. So, let me propose this example. So, typically when you join an organization, um you'll get this org chart, right? And the message here
is clear, right? The CEO is the top. The power and influence kind of flow down all the way down to the individual contributor. Um but imagine if you joined a owner organization and they gave you the social network chart. You would see something that is really really different. Uh I would say that this is how actually gets done in a company where uh individual contributors can have
an enormous amount of influence and it's not just uh the CEO at the top who's deciding everything. So for me using social network analysis has really given me a new lens right and a new lens means that I can start to ask new questions. Some of the questions that I've started to ask uh or have been asking in the past 2 years are what causes social connections
to drop? Can we detect work and context silos? Uh can individuals be too central to the network? And I think after two years of really focusing on this and really doing it, I think I actually have the answer to a lot of those questions. So what's my methodology? So my methodology started uh I developed it around two years ago in January of 2024 and I created uh
scripts that are on my GitHub repo and you can go download it. I'll share the link afterwards as well. Um and what these scripts do is that they pull down uh a group of people's um social interactions on GitHub. So and then save them into an a CSV edge list. So you have usernames uh interactions and GitHub resources. And the way that this data looks is like
this. So uh you have a GitHub user A for example connects to pull request one uh via a PR that they created. So this is a bipartite network because it has two different types of nodes. A GitHub users on one end connected through um GitHub resources like issues or pull requests or discussions. I also like to look at the data via um this other type of network
that you can get by collapsing uh the bipartite network into a uniartite network. So here you can see uh you can connect people directly to people via uh kind of not only the resource but the action that they took. And both of these are really interesting ways to look at the network. The first one because it really shows you what people are working on. You can kind
of see the resources alongside them. And the second one because it shows you who people are connected to. And uh with this one you can see really clearly uh where the clicks where the groups are. So how do I visualize a network? So I use all sorts of tools. I use things like gey uh viz network, cosmogorph and neo forj. Each of these tools um have really
different reasons why I like to use them. A neoforj for example has a really nice query engine that you can get into the really nitty-gritty of the data. uh gey or cosmogorph have really elegant uh visualizations that you can zoom into and see uh the details as So computational network analysis in addition to the visualizations I also uh uh compute the network every two weeks. So I
compute metrics around the network and things that I capture are the number of nodes the density of the network and more most importantly the connectivity. So what you see here on the red line which is a click this uh laser worked but um yeah what you see on the red line is the mean node connectivity of the people uh of my team members that are in the
network. So you see it uh eb and flow go up and down and this topology tells me a story of what's happening on my team throughout time. Uh, and it's a really interesting narrative that me as a manager uh, who's also doing one-on- ones, retros, uh, can interpret and make adjustments along the way. So, what are what are some pain points that I was able to see
with not uh, with social network analysis and what are the mitigations that I applied? So, there are a couple uh, there were team off and onboardings, uh, clicks and silos, um, seniority bottlenecks, right? Uh this answers a question of can people be too central in network and I think they can. So and also uh myself manager so team often onboardings. Uh so I think it's pretty well
documented that when you add a bunch of people to the team and you think that the velocity is going to go up it actually drops. Uh and this is because you've just completely changed how the team is working and people have to relearn to work together. Well, the same exact thing happens uh with the uh network connectivity. When a lot of people join the team, the network
connectivity completely plummets. And you see that happen uh in these two squares, right? And one the first one four engineers joined and then later on three engineers joined. And it totally messed with the network dynamics of my team. And this also was corroborated by anecdotal evidence with people saying like uh like they're new people, I don't know how to work with them. So what do we do
to fix these to fix this problem? So the first one uh the first things that I do is uh I set up buddy systems and round robbins. Uh buddy systems are finding uh like a person that you can talk to as soon as you join the team and round robins are having the new person talk to every other single person on the team. This may seem a
little timeconuming but what I found is that if you introduce people right away and start them talking uh these are conversations that are going to persist uh later on and help uh kind of people connect. Uh the second thing that I do is I run questions of the day or game days. And initially I was really against this. Uh and I don't know if it's because I'm
American or just culturally I or because I'm a manager and I just wanted to get straight to the meeting. Let's get from point A to B. Uh but one of my staff uh engineers from Spain was like no like hold on we have to have a little bit of fun you know before we start the meeting. So I I took him up his advice and I've become
a really big fan. What I noticed is that if you start off uh a meeting or like a uh a retro or any any type of meeting with a really interesting question or funny anecdote or even play a little game, what it does is that it gets somehow the talking juices flowing. And I realized that people are way more willing to talk later and then collaborate after
the The final mitigation is just don't do it. Uh so if you're in a position to add a bunch of people to your team, uh just say no. or if you're a director and you're planning on adding like three or four people to another person's team, just don't do it. Uh it can be really disruptive and I found a way better way to do this is to
just slowly add one person uh at a time uh to make sure that your velocity doesn't drop. The next pain point was clicks and silos. So this is an example of a graph of something that actually happened. Ah I needed a black screen. So uh the green nodes that you see here are my team members. Uh I'm obviously at the center here. Uh so what's clear here
is that you have a click, right? A silo, a group of people who's only talking to those other group of people. Um initially when I saw these clicks and silos, I immediately remembered kind of my agile training and thought it was a bad thing. I thought, "No, everyone needs to do what everyone else does, otherwise it's uh maybe not a really good agile team." Um, but I
after I thought about it and thought about it for a while and talked to a bunch of people on my team, uh, they said, "Maybe I don't want to know about this other complicated problem happening in this space while I'm trying to solve my complicated problem." So, the first question that I ask now is, is this click or silo bad? Right? Is it detrimental to the team?
In many cases, it can be really positive, right? You can have a group of six people working on a really difficult problem and they don't need to know what the other six people are working on at all. So in that case uh I keep it I let it grow and sometimes that means that your need needs your team needs to split in two and that's completely a
completely legitimate way to deal with it. Other times uh it can be detrimental right and that's when you have one or two or three people that know all the context and that you start and you start to hear uh people say like oh I'm being blocked by this. I can't move forward. So what do you do in these cases? Uh so you do the same exact thing
that I told you not to do in the first part. You rotate people, right? And when you rotate people as you learned earlier. Um it breaks the network up and sometimes when it when there's a silo, you do want to break the network up. So you can apply a a team rotation uh to change the how how a silo is operating. Seniority bottlenecks. So this these are
these were a little bit more difficult for me to spot. Uh but I started to see them later. So um what you see here uh in the center of this network are two nodes that are really really well connected right and then two nodes that are uh on the outside and I'm talking about the the purple ones not the orange ones. So the central two nodes uh
they were actually some senior engineers that had been on the team for like an year and the two on the outside were fairly junior engineers that had just joined Um, so I started hearing uh in one-on- ones, in retros that the two junior people weren't feeling like really included. They really didn't know how to participate. And I feel like this is a problem uh a problem in
a lot of places, right? Uh think back to uh the Phoenix project, right? Where you have a senior engineer that kind of runs everything and everyone else doesn't really know how to participate. Uh or also where you have a really senior a couple really senior people and the junior people like really don't know how to engage the rest of the team. So um what are things you
can do to kind of break up uh a dynamic duo or like a senior bottleneck? So the first thing that you can do um are mob sessions. So for those of you who haven't done these before, mob sessions don't involve any sort of crime, right? They involve a group of engineers getting together to solve a single problem. Um, I do these mostly with customer escalations because I
think they are general enough for a lot of people to handle and they're really informative because they're, you know, what the customer wants, uh, to see in the application. Um, so, uh, if you have a really really strong facilitator that can get people to participate, uh, kind of pass the baton along, you'll find that, uh, mob sessions are a great way not only to get people talking,
but also for senior engineers to kind of show their technique and show how they, uh, approach problems. And what I found is that by consistently doing these mobing sessions, I saw the level of the junior engineers kind of improve uh quite rapidly week after week. Uh the second thing that I did was run junior only task forces. And this is something that made my senior leadership really
really nervous, right? Because you don't want to put a bunch of junior engineers on a project. But uh ironically, this does work pretty well. So, uh, one thing that happens when you have a group of junior engineers, right, is that their goal is to become more senior, right? But they can't become more senior unless they're solving senior problems and they can't solve senior problems or act senior
if they don't have if they're not given any senior problems. So, uh, I kind of flipped it on its head and and said like, well, let's give them a senior-like problem. So what I started doing is I started finding these projects that were fairly complex but maybe not time bound or uh not relevant to like the main main goal of the company and started having groups of
junior engineers just tackle them, right? Um and what I found is that these junior engineers uh over time they started to develop like a more senior attitude, right? Because they were held accountable to the outcomes of that project, right? They were held accountable to the communications, they were held accountable to the timelines. [music] So when you give people when you put people in this [singing] situation uh
it kind of forces them [music] to level up fairly quickly. So did these mitigations [singing] work? So these are real results from the same team, right? The first one uh was um when people started complaining that they couldn't participate. The second picture is just one and a half months later. So that's uh a lot of mob sessions later and a whole project later. And what you see
here is that you can't even tell right who's a senior, who's a junior. uh they all kind of work cohesively as a as a combined team. So that's how I helped to level up some of some of these junior people. What about manager model? So this one was definitely the hardest one for me to >> Collecting web data and I wouldn't have noticed it wasn't for this
complex infrastructure system is uh >> heavy data loads break whenever I go on vacation. >> Meet us at booth 0. Second time it happened, I came back infrastructure for web data. Did people just stop when I went on vacation for AI and I realized that that wasn't true like people actually were working maybe they weren't talking as much but they were 77 million IPS >> and I
wasn't really sure what to make of it but I I took a lot of time to think about it and I realized infrastructure >> this wasn't a team problem this was a management problem and the problem was me right uh what I was doing was I was uh getting too involved in technical decisions like really not really not [singing] leveraging u management correctly uh So what I
started to do is I started [singing] changing my management style and this is really hard to do because I was a fairly competent engineer. I wanted to participate in the engineering but I also had a lot of [singing and music] people to manage. So I needed to do that better. So I started focusing on different things. I started to focus on why like why weren't we doing
this? And I started just every meeting I was like we're going to do this because of da da da. And I started holding people accountable not to the process or not to their technical solution but to the outcome. Right? When I started doing that, I realized that week after week, uh, people [music and singing] started to develop their own compass, right? So, they didn't need um like
these little issues or or tiny little guidances or like for me to tell them, oh, stop doing this. This is not important. People just had a sense of what they needed to do at the company and what was important. So, I could back off a little bit and allow them to explore different technical solutions that I would have never thought of. Uh, so I think that turned
out pretty well. After that, I started to encourage continuity. So instead when somebody would go on vacation instead of saying let's wait for that person to come back and make a decision I just said whoever's here together we're just going to make a call and go with it and if it's the wrong one well you know tough uh we'll fix it next week >> collecting web data
>> I started to do that myself to just go and have the meeting when I wasn't there like most senior staff person wasn't there I said like let's just try to solve this problem today we'll find out what happens later and I think it worked pretty well it started taking away the idea that we needed for Phoenix projects and people databach of looking at the data really
closely and making these corrections started working right >> cuz in the at the start you see that the lows are much lower than the lowers later on and the highs turn into much higher. [music] So if you put a trend line on this red line, you'll see [singing] that there's a trend upwards. [music] And I also heard anecdotally from the team that they felt that they were
working better together. So So I also want [music] to talk a little bit about ethics, the ethics, AI, and [music] cuz I think it's really So uh of course I tried doing social network analysis with AI AI and I'm not I'm not going to lie, it was really cool. So what I did was I took a Neo4j [singing and music] database, dumped all the data I had
collected that year. I first analyzed it of course uh [music] got an MCP server in front of it and then started asking questions and it served as a second brain right I could say what happened last month uh with this part of the team like was this project well connected or not and within seconds I could get all sorts of really interesting and thoughtout answers as to
what was happening. So uh I still use this. It's really cool. I uh I have a lot of the code of how to do this on my uh GitHub repo. Um but I took a pause here, right? And the reason for this is that if you've ever built software with LM, you know that it's really easy for things to just competition. And a lot of what we're
learning what people are working on is we're learning how to add boundaries to the LLM, right? We're learning how to how to say like do this, don't do that. We're learning how to give it like really explicit 0021 and I realized that maybe I need to give myself a little bit like an AI around how I was pricing and market trends reliable business intelligence. So uh one
moment where was when I asked who I started to call engineering manager clap infrastructure to give me a list of the lowest uh ranked collaborators and it did right really quickly and when I cross referenced to the pe to that to the people on my team who [music] were currently struggling it lined up perfectly. Uh so I started thinking to myself it's like huh [singing and music]
could this be a performance metric? could I start collecting all this data and just basing our bonuses [singing] for example on this like could I could this be a shortcut to my management um and then I thought about it and I went back in time. So remember [music] how I said that social network analysis was uh created in the 30s, right? And at the time people were
really excited about this new technology and like new meth research methods and how they could be used. And then other people were saying like uh like maybe if you turn people to data you're going to dehumanize them and maybe there's a lot of bias and those those ideas still persist today. So I I thought about it a lot and I said maybe maybe not, right? So I
added uh a bunch of personal ethical principles for me to follow when doing social network analysis. The first one um is transparency right? So I think people should know uh that their interactions are being used and how um and I did [music] this on my team every two weeks I would share a little graph right uh a little CSV of like [music] what the data was. I
would share the metrics the insight for the team should public internal equals complex infrastructure. I also advanced system always communicated that this was for group research not even though it would seem like the data isn't biased what you're doing at GitHub you're using GitHub 19 there is a lot of bias in the data fast adapting. >> This is where I want to add some of those complicating
factors. So, and show you that the context outside of the data really matters. So, there can be a lot of reasons why you're uh uh an influencer or really well [music and singing] connected, right? Number one, you can be a leader, right? You can be someone who's doing a lot of glue work and that means your centrality uh [music and singing] for example is really high. Or
you can be doing a lot of low value work. So, for example, if you pick up an issue, you comment on it and you're like, your comment [singing and music] is like, I can't finish this. You put it down, like that increases your connectedness. If you pick up a PR, submit it, and it has a lot of comments on it, and then you close it, your connectedness
is also going to be really high because a lot of people didn't like it. Um, it's still it doesn't mean anything, right? You're you're still producing low value work. So, you can't use this as an individual metric in that way. There can also be a lot [music and singing] of reasons for having low connectedness. You could be on vacation for a week or two and we want
people not to burn off. So vacation is important. Uh you can be doing a lot of deep work and research. I found out that when my most senior engineers were doing some really deep research on how to do something completely new um they were not uh participating in the network enough and maybe that's fine, right? They needed heads downtime. And then finally [music] you can have interpersonal
conflicts, right? Someone can have a familiar conflict. Someone can have a conflict on the team. And each of these situations uh kind of presents presents a different web data shouldn't be this hard scaling equals complex infrastructure data and applying blocks without kind of getting that context. >> Heavy data loads break pipelines can fix this. Um in the end uh I did use infrastructure closely and I made
my data engineering management practice a lot. It made me realize things about myself. but one thing a couple things a lot of things that I couldn't explain were like why did the junior engineer after months of being silenced on a on a programming session finally decide to speak up right or why was someone silent for 3 weeks so all these things are um you [music and singing]
really can't get from social network analysis and that's why I always contend that no matter what the network always needs a man manager. So speaking of [singing] um managers, I want to thank my network who helped make this possible. Uh [music] first of all, the GitHub notifications and chat integrations teams uh for being [singing] open to my experiments and giving me the data [music] to analyze it.
Uh Rebecca Miller Webster, the founder of Bright Speak Code for helping me kind of shape and improve this talk. and my director directors and mentors at GitHub, Dan Hoto's, Eric Walker, and Sebastian Lock for encouraging me to continue to try these weird experiments [music] and [singing] also tolerating my obsession with social network analysis for 20 years.
