Get Aligned or Stay Stuck: How Experiments Unlock Team Potential

This talk covers how experimentation can unlock team potential by encouraging collaborative problem-solving and continuous improvement. The speakers, Britney Akre and Corey Nland from Last Call Media, share their experiences in implementing experimentation to enhance team norms and improve delivery processes. They discuss the importance of a safe-to-try environment, where team members can propose and test small changes without fear of negative consequences. Key aspects include establishing clear goals, fostering psychological safety, and utilizing effective decision-making frameworks. By emphasizing a culture of experimentation, teams can enhance adaptability, drive progress, and ensure diverse perspectives are valued in the decision-making process.

Hi everybody. All right, so I think you've probably heard us already say it like four or five times now, but if you haven't already, and I know that you haven't because in that little box it says nine, and I can count so I know for a fact that there are more than nine people in this room. So if you can go ahead and join the Slido, we'll

leave it up for just a second. If you're having any issues, you can also go to slido.com and use that number if the QR code doesn't work. Um, all right. I have no idea what the experiment 16 for is but let's see what do we have. What are you interested in most learning today? How do we carve out space and build buy in? Okay, great. We're definitely

getting part of that. Ensuring experiments are just that. Yes. >> How to get my team using it? What tooling are you using? How to encourage feelings of ownership. Okay, awesome. It sounds like you guys are all in the right spot because we are going to cover all of those topics. >> Anything you want to call out, Britney? >> No, these sounds really great. You could you could

write our talk for us. So that sounds sounds awesome. >> All right, we'll get started. >> All right, so welcome to get aligned or save stack. How experiments unlock team potential. Presumably this is the one you want to be in. >> And if you're not, too bad. >> We're going to call you out if you walk out now. So unfortunately, >> I am Britney Akre, director of

account strategy at Last Call Media. >> And I'm Corey Nland, uh, director of product and delivery, also at Last Call All right. So, um I think actually you guys probably know what we're talking about when we discuss experimentation, but I think one of the important things is to be like, why do Britney and I care about this so much? And the reason why I care about it

is because I know what the results look like. I've been in project product management for a very long time and I've seen what experiments can do for teams and for the things that they produce. So um experiments can be used in a variety of places. It can be used for team norms or like how does a team collaborate with each other and it also can be used

for like processes. So how do we do the work? How do we improve on the delivery of the work that kind of thing. These two snapshots are from uh monthly retros that my team was having. And you can see over six months we are norming, right? The team is starting to have a much more unified experience than they were six months prior. And this is very largely

based off of experimentation within this team around team norms. And here is the process side. So this is team velocity over three years a nearly 240% increase. And this is all tied to the way that we do the work. And what's interesting about this, which I love, and I and FYI, I love this team. Some people who used to be in this team are in this room

right now. So, hi guys. Uh, is that this was very collaborative and we didn't change how we sized items. So, when you're thinking about like how did velocity increase, this is not being manipulated by changing the denominator of saying like, oh, we're going to have a bunch of tiny stuff now or whatever. Everything stayed the same. It's just experimentation on how we do things. >> Okay. So

let's start start talking about how we do this. So from stuck to starting. So what is experimentation? So experimentation is testing small timeboxed safe to try change to learn something specific and then using that thing to decide what to do next. So this is not taking on every problem at the same time. This is about deciding to become actionoriented together. It's about getting just enough information to

take the next step together confidently. So what does it mean to be safe to try? Because that's an integral part about experimenting together as a team. Safe to try is let me check. Hear me. safe to try means that the risk of harm is low and it's reversible. It's about uh the scope being small enough to stop to pull the andon cord uh if needed. It's about

uh the consequences of failure being learning, not harm. So safe to try is is that the team agrees that the thing that we're doing is just worth attempting. Um it's that not that we all think that it's it's going to work out completely. It's going to work out 100% successfully. It's just that it's we're just willing. That's it. It's it's just a a tiny step forward that

we're all willing to try together. And it's something that we can live with. It's not that we're all agreeing everything's perfect. It's not that we're ignoring all of the risk involved. And it's not that we're guaranteeing success. It's just that the downside is survivable. So, we're just really talking about like a barrier to entry. Um it's that learning is worth the attempt. And so you see like

all of the all of the things I'm saying is like the least viable thing that we can do together. Um because it's just biased towards action. >> So what are the goals of experimentation? So we're talking about 15% better. That's sort of a liberating structure. I'm not sure if anybody is familiar, but you should be. >> If you know what a liberating structure >> Yeah. I I

love that we've like worked with everybody that not you though. All right. Yes, for annoying liberating structure. Um, 15% better is again like we're not saying we're taking on everything at once. It's this idea that uh what we're trying to do is take achievable bite-size steps together um in just an effort of like just forward momentum 15% better. The next thing is just finding long-term solutions to

reoccurring reoccurring short short-term problems. So these problems that come up every single sprint, every single day, and we're never finding solutions to them. Experimentation is the idea that we can find a solution. We don't have to keep dealing with the same problems over and over and over. We can find a solution and at least deal with that now. And maybe that changes later. Maybe in a year

that solution isn't working anymore. We find another one. But for now, we can find something that works for now. The next thing is that um it accelerates adaptability. So here's the thing. Teams that um implement experimentation as a norm, you also find that uh they become more adaptable. They find that they pivot easily. Um so it just that also becomes a norm for them. And then my

favorite part um is that you also find that they facilitate experimentation facilitates diversity of thought. And the way that it does that is that experimentation changes who gets to contribute to solutions because we start testing ideas instead of debating them. It means that the best ideas are the ones that we go with regardless of where or who they come from. So something that I'm sure we've all

experienced is uh the loudest people get heard. the loudest uh people their solutions get implemented. when testing experimenting becomes the norm of the team then the the best tested ideas are the ones that become our norm and that tends to be uh that's diversity right that's diversity that's equity that's that's what we see in action so what are some symptoms that show experiment could help uh so

let's talk about this the same debate debate keeps surfacing in meetings without resolution do how Often do we see this? Constantly, right? Um, another one that we see constantly, you guys tell me. Teens wait to be told what to do instead of coming up with their own solutions. >> Constant frustration, right? >> All right. That one's popular. >> Popular. Another one. The team doesn't know which of

their practices are working and which ones aren't. Do we experience that? >> Mhm. Uh, how about this one? I used to work in government. I don't know if you guys do. Uh, new tooling or new ideas get thrown out immediately and dismissed because that's just not how we do things. Anybody? >> Also popular. So, these are things that show up constantly that experimentation could absolutely help with.

Now, these aren't personality problems and these aren't necessarily team problems, but these are absolutely problems that teams have a responsibility to deal with. Okay, so before we get to the how, and we were going to get to that in detail, there's a couple of like background topics that we need to cover first because it feeds into this whole experimentation process. >> And the first one, >> oh

listening, >> okay, is who decides what we experiment on and how we're going to experiment on So up here are four very common types of decision-making frameworks that exist in teams, right? We have the autocratic one. We have a boss where it's always their way goes and they're not really asking for advice or anybody's opinion on what is happening. We have the advice process which is a

step down on the autocratic scale where someone is going to make a decision but it's consultative. They're going to go to people who might have information. They're going to say, "What do you think? like this is what I'm planning to do at XYZ. Then we have consensus which is um for example like at this front table if they were going to make a decision every single person

at that table has to agree that that's the decision they're going to go forward before they go on with it. There is no um well only one person wants to do it but the two other two don't can't do that. And then we have consent which is everyone is participating but we're going to consent to doing an activity. it is not going to be consensus based. There

could be someone who does not support it, but we're going to move on anyway. So, within experimentation, my recommendation would be to go with one of these two decision-m styles. And that's going to be largely dependent on how your team functions. If you have one person who is clearly the boss, and that's going to be how it is, then we want to move closer to this advice

process. And if we're more of a self-organizing team or more of like a everyone participates kind of culture, we're going to gravitate more towards consent. So advice is really good even in the teambased structure when the experiment relies on subject matter expertise or skill-based expertise. So for an example, um Britney works more with clients than I do around like how you guys interact. I am not necessarily

the best person to create an experiment that would impact clients because that is not uh my subject matter expertise area. So I wouldn't do that. Britney might consult with me as someone who delivers work for projects but she may not want me to be a designer of an experiment that affects her. Whereas consent, if there is no skill-based barrier, meaning you don't have to be like an

engineer or something in order to have an opinion on it, you should go with this one. Particularly if it affects everybody in the team. So when we were looking earlier at that um snapshot of the team norms on the retro where everybody's moving closer together, that affects everyone. Anything that's in that sphere really should be consentbased because everyone is going to be impacted by it. What's important

between these two is that the way that we handle objections is different in advice process. So you might get a piece of information that is an objection as in oh I don't think this is a great idea for XYZ reason but the person who's going to make the decision gets to decide how big of a deal that particular objection is whether or not it materially affects the

experiment. Whereas in consentbased decision-making, in order to move the decision forward, you cannot object unless it is unsafe to try. And if you go back to what Britney was saying, safe to try means that there's not going to be any like severe negative fallout from it. So someone just not liking it or would prefer to do it a different way is not an acceptable objection within this

model. It has to be unsafe or we move on. Now, if it is determined to be unsafe and you're using consent based decision-m, the person who objects has to be part of resolving it. So, you can't like be a negative Nancy and then walk away. Like, you have to participate in resolving the problem if you notice that there's a problem. All right? So, besides decision- making, we

need one other thing. You can go and it's psychological safety. So this is its own topic. It's got its own thing. But psychological safety is essentially um the belief within a team that people can make mistakes, they can contribute, and negative effect to that. So if they're wrong or if they get it slightly off or uh there's like some social issues like we're trying to resolve this

with psychological safety. This is everybody showing up and trying to do the best that they can on the assumption that everyone's acting with positive intent. And it's a cycle. So it's not like you do it once and then you have it. Uh the what we need here, if you go to the next slide, is we need to manage our psychological safety within the team because experimentation only

works when everybody is willing to be a participant. The only way that you get that diversity of thought, those like new and exciting ideas, is if people feel safe sharing them. But if you have a team, let me let me ask, who has an experience where they wanted to share something, but they knew it was going to be perceived like poorly and so they just kept their

mouth shut even though they didn't particularly enjoy where we were going with that. Okay, so everyone basically got it. Okay, so this is kind of like what I'm talking about, right? You need the ability for people to be willing to share their opinion without feeling like they're going to get shut down because we're going to actually try and discuss it. So, assuming that you figured out what

your decision-m framework is and you are attempting to have psychological safety either because you have it or because you're working on it. Hint, experimentation actually helps you build it. But we can move on. >> Okay. Is it you >> now to the good stuff? Uh, am I close enough? So how do we design an experiment? >> So experimentation is also a cycle. Fun fact, like everything is

a cycle. Nothing is ever completely done. I think we all know that. Uh so when you're in this experiment cycle, there's like kind of five major steps. One is sensing. So like something is wrong off. Something is not working. We need to figure out what that is. We need to sense what is happening. Then we move into design which is for the experiment based off of what's

not working. What are we going to do? And then we test it. Self-explanatory. We retro, right? So we have to understand what we did. We have to debrief on it. And then we iterate. Uh and then that iteration goes back into sensing, right? Did we solve the issue or not? And if not, then we're going to go back into this cycle. We're going to design, test, go

over and over again. um even if you solve one issue, there's another issue to be sensed, right? So, it just keeps happening. >> So, there are a lot of proposal templates out there to help you track and design what this process looks like. But the and the one that we've captured here is going to be included in uh some additional resources that we will provide for you

today. But specifically, what you want to make sure to include are these. Why we want to try it, what you want to try, how you're going to try it, who is going to try it, and then how long they will try it. All right, here's your next question. We're going to give you a couple seconds. And while you guys are filling that out, um, there's also an

open Q&A on that app. So you can either ask your questions at the end or raise your hand or if you don't like public speaking unlike us clearly you can just put it in there. >> Which one of those stages do you find particularly hard to figure out why, how, who, >> why? >> Mainly because we don't always agree on why. >> That's a great point. I

also think that a lot of our environments are very complex, right? So why is often very difficult to narrow down into like one thing, >> right? >> My why, my why is not your why. >> And and you're not wrong and I'm not wrong. >> All right, we'll give you like five more >> Okay. All right. Uh so why is the clear winner with Oh, nope. Last

minute. last seven seconds on the Okay, that's cool. We're going to use this to talk a little bit more when we get to the slides. Um, like I definitely would encourage you if you have a specific question, you can just interrupt us at any not interrupt, but you have my point. You can raise your hand at any point. >> All right, we can go on. >> So,

we're going to go through all of these. Um, we'll spend a little more time on why and how because clearly we have a a preference there. But, um, yeah, feel free to ask more questions. Okay, so how to identify our Y. So why falls into the sense phase that Corey just went over? So the sense phase really gives us time to slow down and ask the right

questions before we delve into the the designing of our experiment. And so the right questions are really about um what do we actually know? So that's what are what are the symptoms that we're experiencing? Have we tried to solve this before? What happened? And then we really want to be honest with ourselves. Do we truly understand the root cause? So, do we know it? But but be

honest, are we are we guessing? And then can we answer this question without trying something? And so that's really important because sometimes when we do have a bias towards experimentation, we want to experiment and it may not be necessary. Are we avoiding a difficult conversation that can answer this for us? and we're trying to avoid a different conversation, a difficult conversation by by doing an experiment instead.

Uh that may answer that for us. Um so that's what identifying our why and and it's it's an opportunity for us to to be honest. Uh does anybody have any questions or anything about our our why friend? They can come later. >> What if the why is coming from something outside of the team? >> I'm going to repeat for our recording. What if the why is coming

for something outside the team? >> The why can come from anywhere, right? It could be from an individual. It could be from the team. It could be from the system that you're operating in either internally or externally. When it's coming from an external source, a lot of times what we need to think about is like yes, it's coming from an external source, but the internal team has

to manage it, right? So what would be an action that we could do internally to try and manage this external this external impact? Um I also had like a thing to share. I don't know if if you have but >> um when she was talking about could do we actually need to experiment on this or are we avoiding a difficult question with team norms? I find this

to be most often the case. So, for example, we're going to make a new rule that the team all has to give each other direct feedback every 48 hours. Now, does that seem reasonable or does it seem like maybe we're trying to avoid having a feedback conversation with somebody directly? And so, we want to force seven people to now do this new activity instead of one person

having one difficult conversation, >> right? Like, I've literally been there. Like, I've literally seen this happen. So, it's really important to think about like not only why are we doing it, but why are we experiencing a symptom in the first place? >> What other questions do we have about why? Because you guys care about this one a lot. >> Before you move forward, do you all have

to agree on the why or can you have different wise to move forward? >> So, repeating for the recording, your question was, do we all have to agree on the why before moving forward? The short answer is no. You don't have to agree on the why. Does it maybe help you later with like buyin? Yes. But that doesn't necessarily mean that everybody has to be on the

same page. And that goes back again to the decision-making framework that we talked about before. Like is your team largely operating on an advice basis or are they advising on operating on a consent basis? Because in consent, it doesn't have to be perfect. It doesn't have to be as buttoned down as it might need to be in an advice situation. >> This one realistically is the hardest

one to deal with. And the reason why is because all of us are operating in complex environments or are people familiar with like the syninfin framework like what what space you're in like is it simple is it complex is it chaotic yeah okay I see some nodding essentially it's like your system that you're operating in may not be a one action one cause has one effect system

right almost none of us are operating in something that's that simplistic. So when we're thinking about our why, sometimes we have to do a lot of like mental exercise to get to a why that could we could form an experiment on, right? Maybe you have to do um teaming uh conversations where you all try and debrief on the issue and try and figure stuff out and then

you narrow it down into something that becomes a testable hypothesis. This step will take the longest, but once your team gets into the swing of like thinking this way, it will go a lot faster, right? Like at a certain point, because this is all a muscle, if you spend enough time doing it, you'll figure you'll be able to figure it out. >> And your question exactly proves

that you are dwelling in a complex. >> Okay. How to designer? What? So asking these kind of questions, uh this is the design phase. So, we're talking about blockers. We're not trying to uh unblock a system. We're talking about like real blockers to uh the exact experiment we're designing. So, um Kelly is going to do this thing in our standup uh and that is going to be

a part of the experiment. Well, Kelly is going to be on PTO for two weeks. So, then um Dan is going to do it in herstead. These are the kind of very real blockers we're talking about. Um so what's in the way? Uh those are the kind of things we're we're thinking about. Uh can we iterate on something that already exists? Uh so what we're asking here

is uh or what we're saying here is that we do not need to reinvent the wheel. Uh the exercise of experimentation is not us trying to um successfully create something from the ground up. We are just trying to uh create better systems for our team. Uh solve things for our team. uh does something a methodology already exist? So maybe we don't have to do an experiment at

all. Um that's what we're looking at before we start designing an entire experiment to solve something that has already been solved and you just have to implement >> or the experiment is trying a new methodology that already exists, right? Because you don't have to create everything yourself. It is not an exper experimentation does not require that you fully design every single part of it and it has

to come entirely from the team. No, you can leverage other resources, you can try other stuff. Um, the can we iterate on something we tried before? This one is very interesting because I find a lot of teams are really resistant to this. Another question, how many times have you heard, well, we already tried that before and it didn't Okay. All right. Again, like half the room. So,

it's like sometimes people will look at this and be like, well, we tried that and it didn't work, so why would we iterate on it? Well, Britney has a great response to that question. So, an example that I run into a lot recently >> is this idea of like, um, oh, let's try a brand new teaming agreement. And it's like, we've done that six times in the

last three years. Oh, did your team change six times in the last three years? Did you get brand new teammates six times? Yes. Then you need to do it every single time you get a brand new, you do a teaming agreement every single time your team changes. It's not that it didn't work. It's hygiene. Teaming agreements are hygiene. So, we do it every single time, right? like

that's that's that's what that is. So, uh sometimes it's not that it's not working, it's that we just have to build the muscle. Another thing that I feel like works uh when we talk about uh instituting a framework that already exists, it's like we can't for the life of us have a conversation. Well, how about we try a circle conversation? >> How about we try these systems

that exist that we haven't tried before? Um we don't have to invent some brand new way. Let's look into something that works and and try that. Also, if you haven't tried a circle conversation, that thing's great. >> Yeah, you'd be amazed how small an experiment can be to have a huge impact. That's the other thing that I feel like people get really turned off on when we

talk about experimentation is they think it's this like big, heavy, laborous process. It's going to have like 45 steps and like, oh my god, I don't want to do it. But in reality, for example, if in your scrums people aren't talking, you can actually solve that with like a 30-se secondond activity and you can try it. So, it doesn't need to be a huge list. >> Did

we have qu This was another one that that had a lot of votes. Do we have any uh questions? Any thing about our what? Okay. It's not your last chance. We have Okay. So, how to design our how. So, this is also part of the design uh phase. We're really thinking about logistics here um and how we're going to um implement this. So again, we're talking about

blockers to our proposed steps. How can we resolve them before we get we don't want to get into the implementation and then realize that we have things in our way. We want to try to think ahead and get them out of the way beforehand. And this is um we're talking about blockers to uh like our actual logistics. We want to think about things uh that are blockers

to like the people. So like who are we informing versus who are we consulting? And we want to think about those are those are actually really important. So people that we are informing are the people that um have no like stake in the game. We want to inform them cuz we don't want them to be surprised but they can't change anything that's happening. People who are being

consulted they may have context or power to change the design of our experiment and we want to consult them before we kick off because in the middle of the thing they will change what's happening and that will cause unnecessary friction for everybody involved. So that's that's really important. So getting that that wrong can really cause uh some pain for everybody involved later. And then also we want

to talk about our metrics before so that we know uh what those look like when we need them. So we want to think about what is good look like, what does bad look like, what is inconclusive like like what does that look like? What does this is really kind of turning out like nothing? What does that look like? So that when we meet it, we know what

what we're learning in in real time. Um any questions about about how we're designing our how >> this is like you guys should all have experience with this right you build stuff so you're all in Jer or something and you're figuring out the tickets and the sequencing this is literally that this is doing that activity um and then uh please don't forget about the metrics when that

applies to whether you're talking about team norms or processes right for team norms the metrics might be well when we retro on it everyone had a good experience or what was exper what was it like explained is that people felt better. You know, that's a metric that you can care about. It doesn't have to be a 42% improvement. It could be, but metrics can also be soft,

right? They can be non-member related. >> That's a good point. So, something that I measure uh very often is like client health. Um, and sometimes people do that in numbers and percentages. I like uh some mixture of percentages depending on what I'm looking. I like numbers of like contract value, but I also like just a good color, >> green, yellow, red. >> Um because uh it's a

very easy indication to people who are not in the weeds with me. Um, numbers often mean something to me cuz I'm in it and I know exactly what it means. But to people who are I have to report out to, it's a very easy thing to say and it and setting that ahead of time so that people have understanding before I get into it is an easy

way to for them to know when I say green later they they already have an understanding of what that is. So, uh, you can set those things to mean whatever. I could say purple as long as we decide on what that means at the outset. Um having that that shared understanding really um helps us later >> How does design design our who in or how long? So

uh this is just who's participating who's affected. Uh so participating the people who are actively uh participating in the experiment and then the affected are who is affected by the experiment and those don't have to be the same people and sometimes there's there's overlap. So participating is who's in it. So this person is doing the thing and then this person's work is affected by the outcome of

the thing. Um and then what is the meaningful time frame? Uh maybe we're doing it for a couple of sprints maybe to see any meaningful change. We we think that we may have to do it for the quarter. Maybe we think we have to maybe have to do it for two. It really depends on what kind of experiment we're we're running. >> Any questions there? This is

pretty straightforward. >> Who and how long? >> We should be attempting to do it for the least amount of time that seems reasonable for the thing though. We don't want to prolong it unnecessarily. >> Would you ever run an experiment where you have some people doing it and some people not just so you have a control of like well maybe this is working because there's more sunshine

this week. So the question for the recording was, would you ever do an experiment where you're including some people, but some people you're not specifically for the purpose of a control group? >> I have not done that before. Usually I'm a part of a team that is trying to reach a goal together. So we've identified something we're trying to solve. Have you? >> I've unintentionally done that

before. Um uh which was that my team did it and the rest of the company did not. And so then I could see whether or not we were doing better on average than they were. And we were um but yeah, like I have unintentionally done it. But yeah, if you have a relatively small closed system, you could do that. It depends on what you're testing, right? But

you you obviously could. If you're if you're testing something though where like everyone is working towards like a unified goal and the experiment would affect like the output, maybe that wouldn't be recommended within the same team. Like I'm thinking like well three developers are going to try this and three developers are going to try this. Maybe if you're really stuck or like the team is really struggling

that might be a good idea because you got to figure something out in the same time frame but if you unintentionally impact one portion of that team like they shouldn't be held accountable for it I guess is what I'm saying. So you can >> usually where I see things like that is more like you know like marketing does AB testing or usually I'm not seeing that in

like team norming or or things like that. So, I don't know that I'm against it. I just know that usually if we as a team have identified a a problem, we've all identified a problem. And so, we're not trying to prolong a problem just to see if >> Yeah, that's >> you let me know how that you try that. You let me know how that goes. >>

Yeah, that's actually a great point because in the cycle, right, the next step after retroing is to iterate on it. So if you tried something, even if you had a control group, like you're essentially like separating out a small group of or or a portion of the affected people when you were going to probably iterate anyway. Um, so it's like, yeah, I could see doing it, but

it would probably be like intentional choice, I guess, like where you're trying to do something very specific. >> Good question. >> Why are some key factors What are some key factors to making experimentation work? So a clear test hypothesis. So uh understanding um what you are trying to prove is really important to starting your experiment and it's really important to understand that you are not trying to

prove your hypo your hypothesis right. It's really just understanding what you are measuring against. Uh the point of the experiment is just to understand where to go next. Not necessarily that that you are right in what you're trying to do. Uh bias toward action over per perfection. um really in all things not just not just in experimentation is is the goal. Uh but uh and then next

is celebrating any learning not just uh your outcome and then reing before you move on. Retroing is so important because that is where we are documenting our learnings and documenting our learnings is so important so that we know where we are going next and if we are not documenting our learnings we will forget uh we will go to the next thing and the next thing and then

somewhere along the line we will forget what we have learned and we will lose that momentum. So it's so important that we uh and and the momentum is the point and so it's important that we that we keep it. anything there. >> Who has ever been told that retro doesn't matter and the team's gonna continue this and we don't care? >> Oh, okay. Um, who thinks that

that's wrong? >> It's wrong. >> Okay, there we go. Okay, cool. Yeah, please retro. Don't skip it. >> retroing is the most important part. >> Yeah, arguably. >> Okay, so where do teens commonly get stuck and you guys are going to tell us first and then we're going to repeat your answers back to you. So, we'll give you like a minute. Um, I don't know. Do you

have any thoughts from the last section that you didn't get to say? >> I have a lot of stories. >> What's one? Tell them a story while they're doing that. >> Oh my goodness. >> Oh, nice. >> Well, first of all, there's a retroing. I work I've worked so many places where >> where teams don't retro. >> Yeah, 100%. >> And then wonder why we don't surface

learnings. We don't have learnings across teams. >> Oh, yeah. That's my favorite one. like we did something. We didn't retro on it and then six months later somebody said, "Didn't we do that? Can someone find a JUR ticket where that said that we did that?" I don't know. We don't have any documentation. Then you just got like three people in Jira just using like 18 keywords to

try and find that one ticket. >> Didn't we make this mistake before? >> Why do we keep making this m Oh, did we? >> That's a good one. Yeah. Lack of transpar not transparency, but lack of like self-awareness of how many times the same issues been repeated. That one is crazy. All of a sudden, you start retroing every sprint and it's like, "Oh, yeah. Wow. Why does

this same thing, why does it come up every two weeks for four months? >> Why did we not know that?" >> Okay, what do we got? >> Hey, >> resistance to change. Yeah, totally understand that. We're going to talk about buying in a second. Low follow through. We're also going to talk about clarity on value. The clarity on value one, that one you will find material improvement

on if you follow those steps, the who, the what, the where, like not where, but why, and how. Once if you've done all that prep work, it's a lot easier to say like what the value is going to be because you've already defined all the things that are going on. Yeah. >> Um, okay. Resistance to change is the biggest and then low follow through. Okay, we can

move on. >> But tiny, we don't struggle with these. Nobody everybody struggles. >> I I love that. >> Eight of us. Okay. So these are four of the biggest buckets of problems that teams struggle with. We have buy in, perfectionism, follow through and iteration. All right. So buy in. Buy in right is the idea that everybody on the team or all of the people who are going

to be affected or willing to try the experiment or at least participate in it. Um if we want to get buy in the best thing that we can do at the outset is to directly explicitly ask for them to participate. Be like hey Jane I would like to run this experiment. It's really important that you are a participant. Are you willing to do it with me? Asking

directly is a great way because all of a sudden people feel like okay I have shared ownership. Okay I'm part of the team. I'm being consulted. Um the other one is to this is big on this list. I'm not going to read the whole thing is to sort of like start with the problem, right? We don't need to like get into like judgments or like why is

this happening, etc., whatever. We're experiencing a thing. Let's just try and solve that thing because if I'm not enjoying it, you're not enjoying it. Whatever. Let's move on. Try not to be like judgmental of the system that's happening. Um, and then I would say co-design it. I personally almost always use consent based decisionm. I very rarely use advice base just because I think people have the should

have the ability to comment even if it's not something that they're going to directly be doing. But co-designing really really helps because people then feel like oh yeah my opinion matters which is again a psychological safety thing. If you have it you will not struggle as much with experimentation. And then perfectionism we all know what this is. The the biggest one that you can do here is

just say that it's good enough to try, right? We are not trying to aim for the 100% solution every single time we experiment. We're trying to aim for the 60% the 70% cuz 70% is much better than nothing, right? So, we got to we got to move that out. And if you're a type A person or the kind of person who really struggles with this, you can

also just name it to your team. Be like, "Hey, like this is my whip. Um, I don't think it's perfect, but I want to put it in front of you. And if the team is like, "No, this looks good to try." Great. Right. You're getting other people's involvement, and you can move All right, we can keep going. All right, follow through. So, follow through has a bunch

of different components to it. One of the ones that will help you is if you schedule the retro as soon as the experiment starts or before it starts. So, if you know you're going to run it for, let's say, like two sprints, so in two retros, they're going to talk about it. make the retro board or whatever you're going to use. Make sure you put this topic

on it and be like, "Okay, cool. We are gonna talk about it." Um, check in with people who are participating while it's happening. It doesn't have to be like a big formal thing. Just like slack them or whatever and be like, "Hey, how's that going?" Cool. And then move on. Um, and you have to always document your results. So, we keep talking about this, right? Like you

have to share what happened. It does not matter what the outcome was. You have to write it down. You have to put it somewhere. And we'll talk a little bit about how to do that, but you have to do that. If you don't close the loop, then people are going to continue to have issues with follow through. And then iteration, we've already covered this like 8 million

times, right? Retro. But the point of that is to say, are we going to keep what we did? Did it work? If it didn't work, what are we going to try next? Or are we going to abandon it because it turns out it wasn't a real problem? Like maybe you misidentified it. That's totally fine. All of that information is valuable, but you need to what how it

went in order to decide what you're going to do next. >> Oh, yeah. Go ahead. >> Also, abandoning it is a decision, >> not neglect. >> Yeah, this is intentional. This is you saying, "Oh, I thought we actually had a team communication issue. We don't have a team communication issue. We actually have a client expectations problem. So, we're going to abandon this experiment now that we've tried

it. We're not going to continue with it. and we're going to create a new experiment that deals with the client expectation management stuff cuz that was the actual problem. All that's good to know. Any final thoughts? Yeah. No, I'm good. Okay. Oh, this is you. This is me. Uh so, uh Adam Savage says, "There is really no such thing as a failed experiment. Any test that yields

valid data is a valid test." So, all learning is good learning. The goal is learning and not being right. And so here the fear of being wrong is often what keeps a team from trying to begin with. And the thing the thing is the the fear is common and and that's understandable. The problem the h our hypothesis being wrong is not the problem. The the problem is

that that keeping us from doing the work is really just another kind of failure. And that's what we're trying to avoid here with this us teaching each other and having this conversation about experimenting. And what we really need to focus on is the idea that there isn't a way to fail experimentation except not doing it. And so what we really want to walk away with is this

idea that any data that we learn from experimentation is something that we've learned. And what is the goal is just to inform ourselves about how to make the next decision. And at minimum what it is teaching us is what to do next. At minimum it's helping us to stop debating whether the thing we were trying to decide like is that useful. Now we've learned either it is

or it isn't. And now we can decide whether to keep doing it or to stop doing it. And so next we just have to ask ourselves now that we've learned of this new thing what do we do with it? And the answer is we iterate. Okay, so how do we maintain our findings? This is the boring part. Okay, you got to standardize the process. You got to

figure out how you're going to write this stuff down. So most of us are going to have or all of us have some kind of knowledge management system within our company, whether that's Confluence or notion or whatever. You need to create a template. We've given you stuff in the additional resources which we'll give you a QR code for at the end. But in that template, it has

all those main sections and it'll prompt you on like what are you trying to solve, right? And then if you standardized it, you always sync it through the same way. You create the template document at the beginning of the experiment and then you update it when it's done. You will have at the end a full library of everything your team has ever tried and what the results

are and you guys can reference that. You can also use it to create like a new practice. I've seen this a lot, right? We tried an experiment, it's great, now the whole company wants to do it. Well, you got to create some instructions on how to do that. All of this will help you do that. Um, but I understand the documentation sucks. Nobody likes it. Everybody is

miserable every time they're told to write anything. I really get that. I would honestly say if that is you or your team, please, for the love of God, just do bullet points and move on. But like, keep something. Don't worry about whether or not it looks perfect, but please document it. The one cool thing about this kind of documentation is that once you finish it, it's done.

You don't have to go back and update it because it ran. It was over. Please. Okay. >> All right. So, wrapping up. Um, experimentation is a muscle. You have to practice it. The more you practice it, the better you're going to be at it. It's the same for literally every other skill. Yes, it will take you a while, but it honestly is truly worth it. every single

team that I've been on that has adopted experimentation as a process has really thrived with it. Um, and a bunch of those people are in this room, so I'm sure that they would agree. Uh, and then small tests can move the teams when a big one is either not feasible or like not manageable. The reality is is almost everything can be solved with a much smaller solution

than you think, right? It's just about getting down to that small solution and trying it and seeing what happens. And yeah, like Britney said, a result that doesn't work, it doesn't matter because you learned something from it. And there will never be a point where you know everything, everything is working perfectly and you never need to iterate again. So you're already in this, you might as well

make it efficient. Okay. So um we'll also do live Q&A if you want to put your Q&A up here, too. But if you that's cool but if you just raise your hand we're al we'll also take care of that. >> Okay the first one is you cuz you mentioned it. >> I love teaming agreements. Uh so every time your team is uh new so the introduction of

a new person you should do a teaming agreement. So that usually consists of like some manual of me. So that's usually like um I'm Britney. I prefer communication via Slack, a quick Google Meet or Zoom, whatever you you use. I prefer feedback directly uh one-on-one with a third party like whatever your preference is uh immediately or you know in a sandwich or whatever whatever your preference is.

And then you'll say something like um what else would uh that would be the the manual of me. And then maybe we will make other agreements as a team. We will assume positive intent. We will agree to be respectful. I like to include other things. It's okay to eat lunch. >> Or or things that are like how do we handle conflict? Okay. Like let's say two people

disagree. Can you have a yelling match? No. Okay. We already know that. Some of us sometimes forget. Not >> Not me. but other people do, right? So, it's like you formalize it, right? And because the teaming agreement is formal and it has like an agreed upon set of like behaviors, when someone is not following that, >> we have something to refer back to. >> Yeah. It's not

you being like, well, Bobby, you're not following the teaming agreement. Like, you're bad. It's like, here's the teaming agreement. Remember, we agreed to this. >> You're not respecting the agreement that we we made together. And the reason that you refer back to it or you make a new one every time somebody joins is because they may add things. uh that you didn't think of. And and I

think that it's, you know, like I like to add things like we can have lunch because it adds a it adds levity, you know, cuz there are very serious things like the way that we speak to each other is important. And then there just things like, hey, I didn't get a chance to eat lunch and I'm sorry I'm on this call with my mouth open, but you

know, like it adds things like that, but it it it's a it's a way of formalizing the way that we work >> Um there also may be things in there about the tools that we use together as >> Yeah. Yeah. So, >> hey, so we've we've worked at a place where there's so much experimentation going on all the time. You know, they send you to NWOW, you're

like a seven-year-old at confession trying to think up sins, like trying to think up experiments and uh you know, so like when you come into that envir environment that's already like that, you know how to do it. Could you say a little bit about what people can do what the things that support that you know how you get high level buy in you know how you create

a culture of it at your >> well if it already ex you so the question was how do you create a culture that would promote experimentation are so you're saying from scratch or are you saying like what are the aspects of the place that lets them do that >> yeah when you're like when you're going into a place where it's not a thing >> okay so when

you're going into a place where it's not a thing what I've done this my recommendation is you start with your team only with things that don't require outside consent or support. So you're not going to touch anything that would then affect a process that some director is going to be mad about, right? And you're going to pick something really small that is easy to do and easy

for it to be successful. So, for example, if you are at a place that is agile and you're supposed to be doing scrum, but for some reason they never get scheduled or whatever and so you nobody talks to each other for two weeks, the experiment might be we're going to have one scrum a week and everyone's going to come for the 25 minutes and see what happens.

You just want to start really really small. If that place is not of an experimentation place at all and just keep it in your team. Don't do anything that goes outside. Once you've built the muscle or the team is more comfortable with it because there's going to be other issues that are going on in an environment where experimentation is frowned upon, um, you can then move outside

of that team and start doing other things that might have a larger impact. But just start itty bitty on straight up. >> I introduced check-in questions. >> Oh, that's a good one. Yeah, just a check-in question. Like people don't talk to each other. >> Wild. Can't believe people do that. we work with each other for 30 hours. >> Um, so every quarter I I sit down with

my team and go through what's what's working and and what's not working. And for a lot of the things that come up, uh, in my head I'm thinking, "Oh, okay. That's that's sounds like a solvable problem. Yeah, we could probably Great." Uh, but sometimes in my head I'm thinking, I don't think that's a what's what's what is your perspective on this? that maybe I'm totally wrong. Maybe

there is no such thing as a intractable problem or if if you come across things that you think areable like how do how do you approach experimentation on them? >> I got that one. Some problems are are universal. >> Oh, sorry. What do we do when we are posited with a problem that we do not believe is solvable? Um some problems are universal and they are not

ours to solve. So often what I will do is I will kindly hand the the problem back and ask what they believe the solution to be. So recently um someone asked me essentially I was dealt with a I I a team member felt hesitant to part participate in solutioning a problem because they felt like anytime they had in the past it was met with like detractors like

people dismantled whatever their efforts were so they didn't want to participate but it was still a problem that they wanted solved and so I don't know what to do with that that energy right so my my response was you've identified a problem that needs a solution, but you don't want to I'm not sure how to help you there. So, you tell me, you know, and so the

response was, I hear you. And so, they chose to participate. >> What I'm describing is is not really about people not participating. It's more like, oh, the problem that you're describing, there's external economic forces that cause this situation to be in in our team. >> To solve that, we need to hire twice as many people. >> That's not economically viable right now. Uh >> in that case,

I think it's just validation. >> It's validation, but the other option that you could take in that situation is symptom mitigation. So if you can't fix the reason why it's happening, you could potentially do something that fixes the ex not fixes improves the experience of the people who are feeling it. Right? Part of that is validation saying I hear you like we are in this space. But

part of it is also even when things are externally affecting us, we do have power to change how we experience some of those symptoms. We can't change all of it, right? Like if a client wants a million dollar car with a $12 budget, unfortunately I cannot give you a million dollar car, but I can, you know, maybe give you an umbrella because what you're really upset about

is being rained on, right? So it's like there might be something else that we can do um while the longer term effect is still >> All right. Um advice for teams where members are wildly different in terms of skill and knowledge. Um, that's a good question. I guess I would say why would that matter in this case, right? Because almost all teams are going to be made

up. They're going to be crossunctional. Not maybe almost all, but many teams are crossunctional with different levels of experience and stuff like that. You can still experiment even when that's the >> I think that's I think that's a plus. >> Yeah. The only thing I can think of is maybe if you're talking about like your team has five engineers and they're all wildly different with like junior

all the way up to super senior, right? And so you're trying to find an experiment that works for everybody. Well, you would need to have those people in the room. I would recommend having those people in the room co-creating whatever the experiment is. If the experiment is like, well, everyone's PRs are terrible and no one comments on anything. Well, your skill level has nothing to do with

whether or not you comment on PRs. You're supposed to do that regardless of your level. So, it's like I don't know if it would necessarily matter. But if the person who asked this had like a specific thing that that doesn't answer, let me know. Um, can you share a real life experiment story? Do you want to do that or you want me to do that? >> Oh,

I have plenty. Uh, we experiment all the time with team norming. So, that's Um, where we try Yeah. So, team norming would be like how we try teaming agreements and the way that we communicate with each other. Um, and so what we do is we end up stabilizing the tools that we use, the way that we document our meetings and our decision making together so that those

don't get lost in the sauce later, right? Like what decision did we make? What did what did we decide that we were going to do about that thing? Um, because then we can call it back up and say these are the decisions that we made and why. um that has been wild that has been wildly successful in everywhere that I've I've worked because sometimes we have hourlong

meetings and we don't we walk away and we don't remember what we we decided there. So that that has happened in many places. Um another one would be uh decision making is often very difficult on operational on on operations teams. Um we meet for hours every week and we don't seem to get anywhere. So, we've implemented um decision-m frameworks and that really helps us get to a

decision on things that we had spent months prior trying to decide upon. Um I think those are those are two examples. >> Yeah, I I had one on more of the like admin side. So, who here operates in a team where you have more than one client? Okay, looks like the majority or maybe half do. So, Who here has a problem with time tracking? >> Okay, we're

over. >> Oh, we're over. Oh, we're over. So, if people need to leave, feel feel feel feel feel feel feel feel feel feel feel feel feel free to head on out. Um, so one experiment that we did at my last shop was we were operating in a service-based model where we had multiple clients. And so there was a question about like which client comes first? If everybody

has something urgent, how on earth do you prioritize that? And we created actually like a visual system that had each client and all of their available hours. So, a person could say, "Okay, well, we have 10 urgent tasks and um this person has decided on their own that they're going to handle these two." And then they take those clients hours out of the pool. It's a visual

representation, so no one else can take those hours. And then they work on the issue and circle back. So, it was like an experiment on like if the if using Harvest or the time tracking app doesn't work, what's another way we could do it? and we turned to the visual and then the visual actually unblocked us on figuring that out. So I'll give you an example. >>

Also, this is so insane. I'm going to turn this off. Um but like we literally implemented voting to make decisions. We started just like thumbs up, thumbs down, abstain and it made like the world. >> We started making decisions. >> Changing meeting formats is also a really common one or like the way that you structure a meeting. just adding like one small section actually can change it

quite a bit. Um I think there were others but there was one that says as a lower level person what if the process changes come from top down and it's not presented as a structured experiment where we're doing this right now. That is not an experiment. That's a process change and they've made it and they did not consult with you. >> Um that's not an experiment. An

experiment is something where you are testing something for a limited time frame. Um, that being said, the autocratic method, if you think back on like what I showed you before, you could technically experiment and it's autocratic, but the person's not going to think about it as an experiment. They're probably thinking about it as in like, well, that's not working. I'm just going to make a change and

we'll see what happens. And I don't really care about the downstream effect. Unfortunately, that's sometimes what happens. I think there were more. Let me see if I can scroll it. Does anyone have any like live questions? Oh, yeah. All Do you want to take any of these? >> Advice for team members who have anxiety about group participation. >> You're going to have to take one small step.

I want to say out loud, I do not have this problem, >> so forgive me. Um, but I think that you're gonna have So, I think um, in a previous talk, my thing was like uh, it was like my call to action was like be a little bit brave. And so, the gold was like don't take on the whole thing, but but start small. Uh, look for

opportunities to take one small step. And so, if you're looking for ways to participate, do one thing. Maybe in Scrum, ask one question. Maybe pick someone that you feel a little bit safer with than others and ask them one question or offer one one idea. Um start there. Um maybe if you feel comfortable ask someone for help in that way. Uh I had someone in a previous

place uh she was actually a senior um architectural what was what was Jess a senior >> I have no idea. I don't remember >> technical architect >> and she came to me and was like, "Oh, I need a a coach." And I was like, "How in the world? I'm not technical." And she was like, "I need someone to help me like speak up and take up space."

And I was like, "Oh, bet." Like, "I can absolutely help you do that." And it was the most rewarding thing cuz I actually watched this person who was so supremely talented technically truly blossom in the way of like standing in her power and using her voice because that's not where I struggled, right? like I I I can I can do that. Um and so I think that

is also a way like if you see somebody who does not struggle with that, maybe that's the way you be brave. Go and say like this is the thing I struggle with. Please please sir. >> Uh hi, I'm anonymous one of them. I'm the top one. Uh I probably didn't write that super clearly, but I have team members on my team that when we try to do

collaboration, >> you mean the facilitation side of it? Yeah. Okay. The facilitation side of this is that you have to make the meeting more um forced egalitarian. What I mean by that is >> yeah you have to implement something that requires participation from everyone. So an example would be a liberating structure called a circle discussion in which everyone they can pass. >> They can pass >> but

it it really does prompt people to contribute. Most people actually don't pass when they're doing this at first, but it creates this norm that everyone's voice is required to be heard in this >> They even have to say the word pass. >> Also, sometimes those of us that talk a lot um >> we have to practice not and letting the silence linger. >> Yeah. So even that

is something and if you're the facilitator, it also helps to go talk to yourself and then also the other people who talk a lot on the side and say like, "Hey, I need you to >> to to pass the mic." Yeah. >> Yeah, that's a really good one. I I also >> Yeah, that's exactly what I was going to say is to make sure the people who

do participate leave space for the other. Just be intentional about as a group talking about that >> and and build the psychological safety. You can also do experiments that are like sort of aimed at helping that specifically. Um >> I can't think of a concrete example right now, but like >> mentor those people specifically. >> Yeah. Like in these cases that's part of professional development. Like >>

yeah, >> that's what I'm trying to do. But like you know, they're just >> innate personalities. >> Yes. that, you know, just have difficulty with this way of communicating. >> And that's fine. It's just like everything else, right? The more they practice it, they'll get better at it. They're never going to be the like they're never going to be like Britney, but that's okay. It's not required.

Like, it's not required. >> Um, a couple of these are related. Is experimentation best formalized into tickets or fixed? Um, and then how does this get represented in a sprint like a spike? Um, generally you're not formalizing this stuff into tickets unless that's how your team does everything. If you had like an internal board where everything that you do internally also gets represented as tickets, that's totally

fine. If it's not and it's just a practice you're going to try, then just stick it in confluence or not or notion or something like that so that everybody knows that it's going on. Um, but there's no when you say formalization a lot of times like an experiment is not going to be something this specific like there's a fixed percentage of work for each person like people's

skills are different and generally in agile we don't operate that way anyway right like people take things when they're free when they finish their stuff um I would say if the experiment doesn't require rigidity don't add it don't add it'll make it harder Um, if your team has a lot of ideas for experimentation, what are tips on how to prioritize? Just meet with the team. Yeah. And

vote. Like she said, if if there's like eight you guys want to try, don't do all eight at once. >> That's a terrible idea. Um, that's too much for people to hold. Like I think even in like my highest functioning experimental team, we were maybe doing two or three at the same time, but not in the same subject. Not not in the same >> Keep it fun.

>> Yeah. and and as least burdensome as possible. >> because you'll you'll you'll burn out if >> Okay, we're way over, but does anyone have any live? >> You guys were so awesome. >> The last session. >> So, do you have any advice or recommendation on how many experiments you think should come from the top down or from the bottom up? >> It should it should come

together. I think you guys should have a conversation. You can make suggestions. Everyone should make suggestions. make it a conversation I would think but I don't think it should feel top down >> I mean it depends on the system right because if they're in advice that's a hierarchical system in which only the top level person decision yeah we don't work that way >> we're not in high

yeah >> yeah thank god it's >> been a long time since we've been hierarchal >> yeah but I mean it depends there's no like straight formula that's better than another whatever works for your team whatever gets you guys to do this is what you should be doing that's literally it. Anything else? >> Thanks y'all. >> You have the template QR code. >> Oh, yes. Sorry. So, this

is our list of resources. Um, like feel free to grab it and then um >> I'll leave that up for you guys to get >> Yeah, it's okay.