Denis Stebunov: Estimates Slow Us Down

This talk explores the debate surrounding project estimation in software development, particularly within Agile frameworks. The speaker shares insights from their experiences in both engineering and management roles, highlighting the concept of the cone of uncertainty, which suggests that initial project time estimates are highly uncertain but improve as work progresses. They argue that reliance on estimates can slow teams down and cause misconceptions, as people often confuse precision with accuracy. The speaker offers alternative approaches to managing project timelines and feature prioritization, such as using deadlines effectively, employing t-shirt sizing for features, and emphasizing regular progress check-ins. The session concludes with a recommendation for project managers to master multiple management tools rather than relying solely on estimates.

They want to know how long it might take to complete a certain project or task. And um some people are pushing back. Uh they say that estimates are very uncertain and there are numerous problems with them and so we should probably not rely on them too much. There is a whole uh no estimates movement in agile. uh there are different talks even books about it and uh

today I'd like to present my perspective on this topic. Um I have been on both sides and engineering and management. Uh I worked with uh different teams uh on different projects. Some of these teams used estimates, others didn't. So I was fortunate enough to see how things unfold uh in different environments. I was uh asking for estimates myself. I was providing estimates, pushing back sometimes and um

so what this whole debate is uh basically the discussion revolves around the idea that estimates are very and it creates a number of consequences. One of them I think is often overlooked, namely that estimates slow us down. They slow the whole team down. Let me tell you how it happens. First, let's briefly talk about the con of uncertainty concept. Uh it exists in uh different industries. So

for example, here you can see a forecast of a hurricane where it might go. Um and uh this cone represents uh the range of possibilities. As you can see, it gets wider with distance. So uncertainty increases. In project management, we also have this cone of uncertainty related to estimates of how long a project might take. But it's reversed. So uh in the very beginning before we even

started the work on the project um the range of uh how long it might take is highest. Uh uncertainty is at its maximum but uh when we start working on it our understanding of the project improves. Uh we now learned the requirements better. Uh we made some progress. uh we validated some ideas. So now our understanding of how long it might take is more accurate and uh

it continues to improve uh until the project completion and so here is our con of uncertainty. As you can see it's widest uh in the beginning. So before we started the work on the project the uncertainty is at its maximum. Uh the cone doesn't have to be this shape. For example, it could be that after starting the work, we realize, oh, oh, that's actually easier than we

thought. And uh maybe after a day or two, the whole task is finished and we are good. That's the happy path. Um or it could be probably more often that after starting the work we realize that uh it's not so simple and uh our prediction becomes more pessimistic maybe even more pessimistic later. But hopefully as we make progress uh we start to see the light at the

end of the tunnel and uh we made progress. We're getting closer and uh now it's done and here is our kind of uncertainty. The whole idea is very simple that our understanding of how long a project might take improves as we progress through the project. So you might wonder how large that initial estimation error could be. And there were studies about this and they say that in

software development it can be up to four times in either direction. So just think about it. Uh if someone says it could take a couple of weeks, it could be anywhere from a couple of days to a couple of months. And uh people are not used to dealing with such huge ranges. it sounds you know it's not useful they cannot do planning with it so they expect

more accurate estimates right and um here some people confuse precision with accuracy uh what do I mean by that um accuracy is uh how likely the target value will be within the range. For example, uh if we say it would take from 2 to 20 days, these estimates has a low precision because the range is very wide. But it's accurate because with high probability, the actual value

will be within the On the other hand, if someone says it takes 4 and a half days, this is a very precise estimate, but it has low accuracy. Um, I personally don't believe it. Oh, someone is concerned in the management. [snorts] All right. So, um what people do when um they have to deal with estimates that are more precise uh than accurate, let me put it this

way. And um developers learn they are smart folks and uh they uh learn to do what? They learn to overestimate and there are natural incentives for them to do so because uh when they have to provide a precise or rather precise estimate there can be an error on both sides but the consequences are very different. If they manage to complete a task earlier, that's great, right? No

problem with that at all. But if they are late, there can be an issue. Even if no one blames them for being late, they don't want to be late themselves because estimates are perceived as some kind of a promise. And uh people naturally don't want to give promises on which they couldn't deliver. So developers learn to overestimate. But if developers overestimate, they must be usually completing their

work earlier than expected. Right? Does it match anyone's experience? It doesn't match mine. And uh there is a reason behind it and the reason lies in psychology. It's called Parkinson's law. The work expands. so as to fill all the time available for its completion. And that's how estimates slow us down. But there is more to it. Developers or estimates introduce some dangerous misconceptions out of which I

would name only one that men and months are not interchangeable. This is a simplified quote from Frederick Brooks from his famous book. Uh what does it mean? Basically, imagine we have a senior developer on our team who is the project lead and the most productive person on the team and they say that it would take a month for them to complete the task. Does it automatically translate

to any other person on the team? For example, to a junior developer who joined the team just recently maybe not, right? maybe they won't be able to complete it in in a few months or fail completely because the task is too complex for them. But uh frameworks like scrum ignore this fact altogether. They pretend that we can plan sprint based on universal estimates and team capacity which

simply is not true. Right? there is a planning error embedded right into the foundation of the framework. How people account for this? They usually just add a safety margin on top of already pessimistic estimates further you know inflating the project timeline. Why they do this? because they want predictability, because they want to show such nice graphs on meetings demonstrating that the project goes according to the plan

and there is supposedly some data baking it up and this is understandable. People want to know that things go as planned but it has its price and the price is slowness. Uh unfortunately predictability in such cases is also sometimes an illusion. Uh I've seen number of situations when things go as planned until a certain point when it became evident that they doesn't. But anyway um some companies

um value predictability much higher than speed. For them it's a reasonable sacrifice. They just want things to be predictable and they don't care that it becomes slower. But if for your team speed matters, probably you should embrace some uncertainty that exists in software development naturally and if you do that you can move faster. I'd like to provide some intuition why uh estimates are so unreliable in software.

I will be making an analogy with construction. Does anybody in the room work in the construction industry? Very good. Because I will probably saying some stupid things I have no idea about construction. But hopefully it won't interfere with the point I'm trying to make. So imagine we want to build such a building. Uh there are parts of it that are understood by us very well. For example,

we built uh the basement and the floors like this many times. Absolutely no surprises there. We built dozens of buildings like this and we know um like relatively accurately how long it would take to build every floor. So these green bars represent known parts. But then there is the roof and uh the roof is rather complex. We didn't actually ever build a roof like that. So we

are not certain. We are not sure how long it might take to build such a roof. Um the overall distribution of time in this project spent on known parts and new and uncertain parts is probably skewed towards working more on known parts because this is how things work in physical world. Even if we know very well how to build such a floor, it still takes time to

build it. we need to break concrete, workers must work on it and so But in software, it's different. Imagine we want to build um an app that does some document editing online. Uh there can be many pretty complex companies there. For example, we have a rich text editor. If I were building such an app, I would probably won't build it from scratch, right? because there are many

out ofthe-box solutions out there. Uh I would probably pick one that fits the requirements better and we are good. What else? Oh, we want to attach files and check them for viruses. That's a pretty complex task. But fortunately, Amazon S3 can do this for us. It has a built feature built-in feature for it. So, I [snorts] can just enable it and we're good. And uh boom very

quickly I assembled a web app with multiple very complex component but it take almost no time from me because of code reuse because I'm not building all these things from scratch. I just plug it in into my project. But now we want collaborative editing. Oh wow. I didn't work on that before. I know that some algorithms exist, but I don't know their tradeoffs and limitations. How well

will they play with the rest of the system? Do we have to change the document format or will it be compatible with the the rich text editor that we chosen? I don't know. That's a lot of uncertainty for me in this part. As a result, in this project, I spent most of my time working on something new and unknown for me because the known part take almost

no time due to code reuse. And this is why estimates in software are so unreliable compared to the physical work because we spend most of our time working on new and unknown things. But despite all their shortcomings, we still need estimates, right? Because there are many uses for them. We uh for example want to know if the project would fit in to a certain timeline or budget

or uh we may need to decide which features to build or uh we may want to set a date when to check progress and that's why we ask how long it may take. or everyone's favorite, we need to manage expectations from our clients, bosses and so on. But if we are using uh only estimates for all these important tasks, um do we really use the best tool

for the job for each particular case? Let's go through them quickly. Uh what if you want to fit a project into a certain timeline or budget? These are essentially almost the same things because in software the biggest expense is salary. So the longer the project goes the more people work on it and the more you have to pay to them. And um timeline is essentially the same

as the budget. Um how do we fit a project into a certain timeline? Use deadlines. Not this way. uh do not ask for an estimated and convert it uh to a deadline because these thing things are uh very different but many people confuse them. Um estimates um assume that the task scope is mostly fixed. So we describe our task and ask how long it might take and

we accept that the time may vary. So we're happy to accept estimates like two to four weeks. That's fine. In deadlines, the time is fixed. We have a certain date when the project must be delivered. And since we have to allow flexibility somewhere, we accept that the scope may vary. But to make it work, we need to do a good job on prioritization. We need to uh

say clearly what is the most critical to ship on that date, what is important, what is less important, what is nice to have. Um maybe some of you have seen this um idea from base camp um approach called fixed time and budget flex scope. uh what they're talking about here is essentially deadlines that uh this is not reasonable to assume that both scope and time can be

fixed. So you must have flexibility if you want uh to ship on a certain timeline. Unfortunately, deadlines have a rather bad reputation. They are associated with stress, burnout, cutting corners and so on. Um however when managed properly this is a fantastic tool for shipping projects on time and without too much stress for the team. Um I'll tell big four my personal uh preferences of how to deal

with First of all, explain why when you set a deadline, you want to get a commitment from the team and people are reluctant to commit to arbitrary deadlines. So, there must be a reason and uh the reason must be real. Um, for example, there is a Black Friday approaching and we are running an e-commerce store. That's a pretty serious reason to ship something before that deadline. Next,

do not set too many deadlines. Deadline means focus and it's very hard to focus on multiple things simultaneously. So, at any moment there should be only one deadline that is the priority for the team and uh don't abuse that thing. Prioritization is the king. If we did a very good job describing what is critical, what is important, what is less important, it greatly increases our chances to

ship the most valuable things on the desired date. And finally, ship often. Don't wait until the day before the deadline to push a faster release to production. uh try to release the first working version of the system as early as you can and then improve it gradually by shipping features from the most important to least important. When all these conditions are met in my practice deadlines work

very well, much better than estimates. What if we need to decide which features to build? The ideal answer would be build features that users want most, right? But we are not living in the ideal world and uh I realize that it's not so simple. There can be situations when our backlog is full of user requests and many of them seem almost equal but the amount of effort

is unknown. And this is why we ask how long it might take. In this case, try t-shirt sizing. Please do not link these small, medium and large sizes to um some date ranges because we are back to classical estimates with all their shortcomings. But uh it would allow um to compare features in our backlog to each other and this relative size is helpful uh to decide what

to build next. A quick tip, if you describe a feature as large, try to explain why it's large. Is there anything we can do to make it smaller? This is often helpful in the decision process. What if you want to set a date when to check progress? Uh what do I mean by that? For example, um someone asks how long would it take to complete this task

and developer responds maybe a couple of weeks. And that person says okay, I will check back in a couple of weeks. Don't do this. This is a bad idea. Just check the progress regularly no matter what. No matter the estimate because when developers work on a task, they usually unpack problems one by one. They encounter some situations for didn't for which we didn't plan for. They realize

that okay this part is taking too long and what we can do about it. When you check the progress regularly, you discover such situations early and you greatly increase chances of this feature to be shipped in a reasonable time. Finally, uh manage expectations. Um, I already spent a significant time on this thing trying to explain why estimates are probably not the best tool for every job in

management, but you probably won't be able to explain it to everyone on your job, right? Sometimes uh you don't have time. Sometimes you are not in the position to explain. Sometimes people won't listen or won't believe you. They just want an estimate. They just simply want to know how long it might take. And [snorts] uh what you can do, remember we uh discussed that con of uncertainty

in the very beginning that uh estimates improve as we progress So if you can postpone your estimate, don't provide it immediately. say, "Oh, give me a day or two to work on this task and I'll get back to you after that." And by then you will have a much better idea of what it would take to complete this project and you can provide an estimate in which

you will have more confidence. So let's quickly recap which alternative tools do we have uh for estimates uh in project management. First we have deadlines. This is a natural thing to use when need we need to fit a project into a certain timeline. We can use the short sizing for choosing which features to build. Regular progress checkins are absolutely recommended to everyone whatever you use estimates or

not. And uh if you really need to provide an estimate try to postpone um your decision. I believe that uh good project managers should learn and master many tools and not just estimates because you know it's simply not very efficient. So that's what I have for today and I'm happy to answer any questions right now or later. Here is how you can find me. >> So bloss.

[applause] >> Thank you. uh also for the future all the questions are going through slido. They will be created by our volunteers and then asked by the speaker. So in data science projects results are not always guaranteed. How should I communicate this with stakeholders with deadlines and scope? >> Uh results uh yes that's uh that's a good question. Thank you. I would say results are not guaranteed

in other areas as well but probably in data science. That's the most you know obvious [snorts] communicate it like straightforward that you cannot guarantee the results because that's how this thing works. Uh data science uh there there is a science there right and you cannot expect guaranteed results from science. uh scientist could you please launch that fusion reactor in you know next year >> maybe so you

would say that frequent check-ins and no scope definition before we can say yes we can build this or no we can't >> um you should be flexible with scope when you need to meet a certain deadline >> like for example if There is a conference happening and I need to come here with my slides. I better come with slightly less polished slides than trying to make them

perfect and being late to the conference, right? The same happens when you have a deadline in in your project, right? You better ship something by that date rather than trying to complete everything and miss the deadline as a result. >> Okay, makes sense. Thank you. >> Sure. >> Next question. Does AI help with estimations? >> Um, I would say it doesn't. If anyone tried asking Chad GPT

how long it might take to build this project and watching the results, I think you would agree. it helps to complete certain things faster which just improves overall progress of the project. Uh but with estimates the [snorts] only um relatively reliable way of estimating work is looking back how long similar tasks take in the past. uh if you did exactly this multiple times in the past, you

can now relatively accurately predict how long the next attempt will take. But the problem is that in software, we mostly work with new and unknown things which simply have no precedent in the past. That's why estimates are so hard in >> Okay, AI is not a reliable helpers there. Thank you. Next question. Estimates are necessary for providing quote for a customer before the project even starts. Any

tools there? >> There are two different models that um software developers use. That's management again. Uh there are two um pricing models uh software contractors uh mostly use. One of them is called fixed cost. This is where you need the quote, right? And uh another one is called time and material when people charge for time spent on the project. And uh risks are distributed differently between uh

the client and the contractor. when it's uh fixed cost all risks are on the contractor because they like signed to deliver this thing by this date at this cost and all risk are there right what uh they may do overestimate there there is no no magic there you have to take the risks they are unknown so all you can do is try to come come up with

some estimate and then put the safety margin there. The bigger you put in better shape you are. Um I personally try to avoid fixed cost contracts. >> Gotcha. Go into consulting per hour base. >> Yes, we we do software consulting and we are not working with fixed priced >> Fixed price contracts. Bad. Great. Next one. Have you seen difference between development and infrastructure project projects estimation? that's

a good question. I didn't think probably probably not. Uh the type of work is uh slightly different. In infrastructure there is a fair amount of planning uh cost management because you know whatever service you use it um is a deciding factor but overall it's the same um kind of half exploratory work as in software. So I would say they're probably in the same bucket. M so basically

a little bit more emphasis on cost on infrastructure maybe a bit more pre-planning but otherwise they are similar. >> Yes. >> Thank you. When we have sprints in agile how to estimate capacity if feature estimate is an 4x or well four times difference uh state. Scrum is a framework built on, you know, some false assumptions. >> The software developers. >> Yes. So, um, again, no magic bullet

there. Uh, all developers can do is basically put a safety margin um because there are risks or things usually not go as planned initially. So all you can do just add the safety margin for yourself and uh that's it. >> Okay. Uh an estimate's worst nightmare is a nightmare client. How do you deal with scope creep? >> Deal with what? >> Scope creep. >> Scope creep. If

you are working uh on the time and material model, that's not a problem. Clients wants more. That's great. we have more work, we'll charge more. Uh if it's a fixed cost, uh probably try to put as much as you can into the contract. Uh you can be flexible later. Like uh you you can say, "Oh, it's not in our contract, but uh your most our most loved

client, so for you will put it there, but you don't have to, right? So try to um fix as much as you can in the contract. >> Okay. So again play with margins nightmare client more margin and iterate >> There are risks and uh basically this uh in this case estimates are about risk >> Okay understood. Thank you. If we do not estimate it means we cannot

plan sprints. So scrum does not work. Question. what kind of methodology should we use instead? I'm not saying that scrum doesn't work at all because uh as I mentioned some deeply want predictability and for them it's like very important that things are going according to the plan and they understand that it it's slower right so in such an environment if you are not in in top management

probably won't be able to change anything but if you have that flexibility uh for me canban uh with accential deadlines works best. Um so yeah >> okay so KBAN is an alternative structure sometimes >> if a project such as a government uh procurement has a hard set short deadline and uniterable deliverables un well unchangeable deliverables. Is there any way to manage burnout in the >> Alcoholism. [laughter

and gasps] >> Um well uh first of all burnout. Um I I'll take a minute to talk about this. people uh might have uh different views on burnout. And uh I saw one um study that shown that burnout happens most in the tip that do not ship frequently. They are working and working and working and do not see their work uh in production or sometimes the project

is scrapped at all. This is very depressing. Uh but uh if uh even if you are sometimes working a lot like working 12 days or or 12 hours in a day uh but the project is deeply interesting for you and you are constantly shipping you are not at at the risk of burnout. Um burnout happens for other reasons. uh and uh burnout is probably um caused by

stress, right? And there are uh certain very well-known techniques for stress management um like uh rest, exercise, travel, uh spend time with your loved ones and you will be less at risk. >> But as a team lead for instance, can you do something? uh watch a team um because uh when you know what's happening right you can see those signs early you may be able to say

oh John maybe maybe you should take a vacation for example or um there is also a very important aspect is the task in hand interesting for the the developer and uh I sometimes I I lead some teams and I sometimes try to balance oh this is a like nasty task uh that I have to give someone but after that I will uh grant you a refactoring for

which you dreamed like all for for a long time and that's kind of you know >> okay so as a short answer here um have a feedback loops ideally shorter if they are not within the project then it's on you as a team lead to include some as well as introduce some gratification small rewards It's to make the work not that monitor. >> Manage the stress for

your team. Some some people are better in managing their stress and some worse. For those who are worse at managing their stress, probably you should do something. >> Okay. Gotcha. Do you see doing the estimates with the team as a good way to share the knowledge within the team and provide the project context for development? That was in the talk actually but I [laughter] thrown it out

because I thought it would take um too much time but uh I've seen some team leads using estimate as a way to validate that um they have a close understanding of the task like for example if I'm thinking that it might might take let's say 5 days and that person is thinking 20 days probably something is wrong right and that's a checkpoint for me uh like talk

more to them uh but it's uh a very unreliable signal. I may think five days and that person could also think five days but we think very different five days and after this five days I can get a very unexpected result and you know we'll have to rework a lot of parts so better not rely on this weak signal just check it explicitly ask that person what's

their plan okay you have this task what are you going to do and as they're speaking you validate that your assumption s are close. I would add on this from my personal experience I don't think it's a good way to share the knowledge but to see the potential problem with domain expertise because as you said some developer might be working on this specific feature for a long

time and they know in depth what's happening and why it takes so long but the other developer although as long and senior uh working in the team might not have the expertise and the expertise is the knowledge sharing which should not happen within the estimate it's just a sign that yes there is a gap in the team knowledge but the question is do you need to share

it do you need to share it in the meeting okay and so far the last question PC's are a tool to improve estimate accuracy I think it's not the question is more of an answer what is the P in >> proof of concept uh yeah they definitely are uh remember that corn of uncertainty what what essentially doing in the very beginning uh is trying to um validate

the most uh risky parts of the project. For example, if we are trying some new technology with which we didn't work before and we are going to build a lot on this technology in the very beginning of the project, we should you know test it carefully with various use cases trying to do the most important parts we want to do with this technology. And this is essentially

our proof of concept. We present it after certain amount of time and say okay it seems to be working. There are you know some trade-offs but overall we prove like choosing this technology for the >> Okay then we have a couple more questions. Why should dev team be willing to commit to a deadline? Doesn't this feel even more vague and unclear to devs than doing some discovery

and giving an estimate? as I mentioned there there there should be a reason why uh deadline exists. Um let me give you a couple of examples. Um for for instance we are running a startup. We raised funding uh for the next few months and this is our runway. uh either we ship something meaningful within this uh time and raise our next round or the startup is failed.

For me, it's a pretty you know serious reason for a deadline. uh or another one that I mentioned is uh some event that is happening in the outside world which we do not control like for example a conference black Friday holiday season or whatever and um I don't know that's that's kind of a business need um and if people are not motivated by you know business needs

uh maybe they should be working on something else. >> Gotcha. And the last question would be do you have any recommendations how to explain to your product manager why you rather would not give an estimate? yeah so basically may I talk to the person who asked this question basically what what's what was your um idea behind it? Are you not willing to give an estimate because of

some reason or you generally don't want to provide estimates? Yeah, sure. Yeah. So, oh you you'd like to introduce them to this concept, right? Okay. There will be recording of this talk. So you can share uh I'm not you know pretend that this is the best talk uh on this topic but there are many others there are even books so uh sure spread >> so try to

on board them on the topic generally but not contradict them on the >> Yeah. So and I I would say that it cannot happen you know just as a flip of a switch. Um so gradually try to you know the very first thing you could try is postpone your estimates like when they're asking how long it it might take very often this is possible to say uh

give me a day or two to work on this and I'll get back to you tomorrow. This is normal and uh you'll get a much better estimate. >> All right, applause for our speaker again. Thank you. Thank you. [applause]