Maik Wojcieszak: About Pink Elephants, Giant Giraffes

This talk by Mike Vesak delves into risk management in complex environments, drawing parallels with his early experiences in software development. He emphasizes that effective risk management requires the recognition of opportunities alongside the avoidance of potential pitfalls. The speaker introduces the Cynefin framework, which categorizes contexts into obvious, complicated, complex, and chaotic, advocating for an agile approach in complex situations. He discusses the importance of data in decision-making and suggests using quantitative methods, such as Monte Carlo simulations, to better assess risks and probabilities. The session highlights the necessity of continuous feedback and adapting strategies based on new information, ultimately promoting a probabilistic culture in organizations to enhance decision-making processes.

[Music] ladies and Gentlemen please welcome our next speaker makoy chesik presenting the topic about pink elephants giant giraffes or how to deal with uncertainties and risk in complex environments welcome everybody let's talk about yeah giant giras you see the image already yeah it's there and uh you may wonder what does that have to do with risk management I tell you a story when I first encountered a

giant giraffe it was back in the '90s when I was was working as a student for a company uh supplying uh delivering U equipment for newspaper production equipment and software and uh yeah I was full of energy and uh Curious and uh I had a side project um just for experiments and created a software to digital uh printing plates I got feedback from customers they liked it

they said it's a good idea we want to buy it I approached my boss and said we can make a product out of that and he turned down this suggestion with telling me I'm not using professional development tools and uh I'm not applying the right automation principles and these arguments sounded to me like he was saying when we do this a giant giraffe will come and eat

us all it does not make sense to me later I understood that at that point in time I challenged his fundamental beliefs he formed during his career a long career and a successful career and these fundamental beliefs uh formed the basis for decisions unfortunately uh this was not the only decision that was not matching the time at that in the 9s and uh the company filed for

bankruptcy bankruptcy a few months later I had the chance to buy back my software and found a company around it which is still in business and we are still selling the software so why do I tell you this risk management is not just what you may think it is it's a great tool to avoid uh that our fundamental beliefs that we formed in the past will create

risk and failure in the present so my name is Mike Vesak which was already mentioned thank you for that and I found founded get next it we are uh helping companies and individuals to succeed in complex environments in in the complex world we have today and the other company I mentioned is V Systems I'm still uh CEO there and Def lead so I'm in the technology and

handson yeah as well so let's talk about risk and chances when we talk about risk we need to talk chances so because they come together and um there are some things you you need to know so if we are having a business we are supposed to take risks it's not something uh that we can decide we need to do that because when we eliminate risk completely we

also to eliminate uh all the so this does not mean we jump off a cliff without uh asking any questions so we want to make sure that we survive the risk and this is why we need risk management because we want to exploit chances so to do a good risk management we need some data data is very helpful you see the sign here uh you have some

data you can make an informed decisions whether you cross the field or not and uh this data is not always available so we need to find ways to to get the information we need for making good decisions so why is it so uh important to make good decisions Annie Duke the author of thinking in bets she's a professional poker player uh put it that way there are

two things which determine success or failure and that is first the quality of our decisions and second luck one of the two things only one we can influence you can guess which one so before we can uh make good decisions we need to think a little bit about the context in which we are making the decision and uh Dave Snowden created um a model for different contexts

which is called the canavan framework and uh in the center of this framework we start with disorder we don't know anything that's disorder and then we try to figure out uh context we make our decisions one of these contexts or habitats which uh they are called in the canavan framework is obvious uh context there's a uh the framework comes with a suggestion how to act in this

context and this is here sense categorize respond so in this obvious context we just look and categor it uh so uh uh we know everything so we we just put it in a a drawer and say okay we we know how to respond in this category uh to make it a little bit more uh uh Vivid let's have a game metaphor you all know the game tic

tac toe you maybe played it when you were in school but you stopped playing it because it was boring if you know how to play it you can't win and uh yeah that makes it very boring I guess that's also the reason why there are no uh International championships for Tic Tac Toe so it's obvious you know everything and you can act perfectly the other um context

is the complicated context now uh we have uh a situation where we still know everything we have all the information but it's hard to make sense of it giving a gaming metaphor for the complicated habitat is chess all the pieces are obviously in front of each player but luck won't help you if you as an occasional player play with or against a Grandmaster because The Grandmaster will

make just better decisions than the occasional player so that's a complicated habitat and now we come to the one which is uh in the headline of this talk uh the context and uh the advice here is probe sense respond it may sound familiar if you're doing Agile development because this is the way we are doing Agile development and agile project management and agile everything test probe s

get feedback respond to what we see and there's one last that's the chaotic context that's a completely different talk uh let's hope we are not ending up in a chaotic project uh I had some in my career uh it's not my favorite situation so just get out of okay besides that we can observe um uh uh strategies how to approach complexity and one is ignoring it you

maybe saw that in your own company or in other companies uh things get brittle things uh break down at the sides and uh people just do the same that they did uh years and years and years and ignore there something has changed around them not uh recommended another approach we try to apply to complexity is a rational understanding the problem with complexity is it is complex that

means rational understanding is not so easy and that means that you may end up in a situation that uh I like to call analusis paralysis so decisions take very long and yeah then the chances are gone probably but my favorite uh approach is simplification suppose um you're faced with the question which tool should I buy for my team which is the best tool team um I saw

people fac facing this question and substituting it with a simpler question because this question is probably not easy easy to answer and they substituted unconsciously the question with the question what is the cheapest that is easy to answer but it's not uh leading in the right direction okay probably you knew know all of these things and uh the good news is we have a gray tool between

our ears that is capable of handling complex situation complex environments and that's of course our brain because it's capable of pattern recognition you may know the pattern recognition as gut feeling or intuition and the intuition is formed uh in situations where you uh in in very tough situations in the past but you can apply them in the present it has some uh it's needs some tricks to

make good use of the intuition because it's slow changing so it does not adapt adapt very fast to new situations So Daniel canaman and uh put it in his book noise uh in great words uh I really recommend this book um when it comes to expert judgment so experts judging on uh based on their experience uh and give information and advice for decisions if you ask experts

you wish that uh the answers are close together if you ask ask different experts or the same expert at different times it should always give for the same question the same an answer but that's not the case we have biases also experts have biases that means uh the answers are systematically uh removed from the correct answer also we have what kanaman calls noise that's for example then

then an expert who had a good breakfast is probably uh less risk averse than when hungry these things are not predictable and uh it leads combined to a an image here of the red dots uh that looks like a throwing a monkey throwing darts so it's absolutely probabilistic and the worst thing is in real time uh real world problems we don't know the correct answer in complex

environments there's no chance to check if an expert is right or not so let's make an experiment expent biases there's a statement on the right side of the slide a qualified project manager improves the chances of software uh of the software project to succeed who of you are project managers uh some you can decide for yourself do you go to the blue door yes I agree or

the red door no I don't agree just remember it and I give you some additional information this information is from the chaos report which was mentioned in the keynote by Gigi thank you for that and uh it uh researched and analyzed uh thousands and thousands of projects and uh they also checked if a qualified project manager uh improves the SU ESS probability of a a project and

uh the numbers for the waterfall project is the result for the waterfall project is if you have a qualified project manager in the project the success is three times less likelihood so if you just don't have a project manager it's three times more likely that the project will succeed and say okay this is waterfall forget it in agile it's even worse if you have no project manager

the likelihood is more than four times higher to succeed so please project managers don't feel offended by that this is just statistical data and we need to explain it we should not ignore it because if we ignore it uh we may fa F victim for the base rate fallacy but if we think about it we may find an answer the chaos report people uh explained it with

uh our human tendency that we yeah feel not so responsible if there is a qualified person which is responsible for the project so the team just not feel that responsible but a project manager can't lead a project to success if the team does not support with full uh energy that might be a possible explanation so um we have these biases we have these problems but anyway we

want to uh make better decisions what does that mean of course uh we need to be better than random so if we make decisions by just flipping flick flipping a coin we get a result um and we want to be better than that so and that's why why we need to probability and yeah of course luck in that context as well a little bit and in complex

situations we have connected probabilities so one question in risk management is does an event favorable or risky uh occur or not it's a binary question we get an answer yes or no so the probability for each is if you know nothing 50% that's not so bad but what is the probability of three heads in a row so that means if your success depends on three different events

to turn out to your favor it's only 12.5% so we need to keep that in our mind and uh think through the different events so the classic risk management appli risk matrices they are recommended in many Frameworks and I would suggest don't use them there are many reasons for that and one is uh based on again how we uh assess risk here on the x-axis we have

uh probability terms from very unlikely so very likely and on the y- AIS we have uh impact terms from NE neg sorry negligible to catastrophic and we sit in a workshop and assess the risk and uh print it into the risk Matrix and then we know because of the color in the background uh we need to act or we uh can just ignore it and let it

go but in my workshops I ask participants Rec uh frequently regularly where do you uh locate the qualitative risk terms on a scale from Zer to 100 zero is uh it will not happen and uh 100 is it is certain that it will happen and I get these results for example for very unlikely 0 to 20% remember the risk Matrix very unlikely and catastrophic outcome uh is

on the upper left corner with a yellow field so we have a an event with 20% likelihood and a catastrophic outcome and we just ignore it that's not a good decision here and the same is with all the other terms there's a wide range that differ from person from to person if you use these terms it's completely hidden uh that there is probably something we need to

care about okay how to change that how to fix that very easy we just don't use the terms we just use the scale from 0 to 100 which by the way is the percentage value to measure probability uh one reply to this suggestion is a precise estimation in percent is impossible because we do not have enough information I'd like to say to this precise estimation is an

Oxon if it is precise it's a fact an estimation is per definition not precise so and not enough information is uncertainty and probability is measured in is a measure of uncertainty so we just measure our uncertainty related to a possible event or outcome you can find uh some funny list in www flaw of averages top 10 list uh of these replies and uh excuses not to use

quantitative uh methods in risk management and risk assessments uh have fun with it I I like it very much so using probabilities uh has another um tricky aspect that is that uh many of us learned in school or in statistic lessons the frequentist definition which is the relative frequency of the occurrence of the event but then you need exact data otherwise you need to work with an

error which is very cumbers summon statistics so we use another definition that's the basian probability and the basian probability defines uh um probability as the degree of belief so we just use the percentage value to express the degree belief you may ask what did we gain in comparison with the um risk Matrix I say you have a number and a number is something you can calculate so

now the bad word statistics I can't avoid it sorry uh when I approach people with the word statistics sometimes I get the reaction of the person in the on the picture uh this is what Sam Savage the author of The flaw of average is called a uh post-traumatic statistics disorder um because hardly uh many people can uh remember their statistics lessons as an enjoyable uh situation and

uh one thing and one reason for that may be that statistic comes with a lot of jargon jargon is a special language a special vocabulary uh of experts or different uh professional groups and it's not really well understandable for uh Outsiders and uh another definition I like is unintelligibly muttering also bird song um so it all it's all related to the problem with uh statistics Jon or

any Jaron to communicate and we need to communicate about um our risk so let's talk about how we approach this problem maybe somebody of you know these formulas uh I did not when I first saw them it's the formula the physics uh when you're riding a bike so you earn all learn to ride a bike without solving the equations I guess and um what you did is

you climbed uh as a child on the bike and you got feedback immediate feedback when you did it wrong so it was sometimes painful uh your parents watched you that you did not hurt you so much and uh after a few tries you you just learned it and then you can do intuitively so why not approach statistics and probability intuitively there's a name for it it's also

from the flaw of averages probab probability management intuitive probability management we need to learn to understand probability and think in probabilities so the first change in your process this is now a practical suggestion estimate probability from low to high was uh earlier now estimate probability from 0% to 100% easy to change now let's have a look at the uh they are not comparable not conclusive not verifiable

and in my opinion not useful it's not a linear it's not a closed scale so we need to look at every risk uh separatedly my suggestion is to use money as a scale if you see this image probably you get a sting in your chest say what burning money it's impossible and that's a sign that we have an intuitive relationship to money so everybody of us has

almost the intuition except you're a millionaire a billionaire if you're a billionaire there is a different relationship obviously so it's clear that we can't find a good estimation on point for impact so how much money it will cost if this event occurs but what we can do is we can estimate a lower bound so it will cost at least this amount of money and we can uh

find an upper bound it will not cost more than and we need to make it in a way that we are 90% confident that the real value is within this interval if so we can calculate things with it and uh work with work with it if sure we can make a simple check uh to work on our biases so if you ourself do I want to bet

that the real value is in this interval th000 Euro of my own money or would I rather choose a Fortune Wheel where I have a 90% chance to win if your cries Take the Wheel Take the Wheel then rethink your estimation you are not consistent with your intuition okay what's more to know about numbers which is impact we know when it happened but we know don't know

in advance that's the definition of an uncertain number and uncertain numbers have distributions and these are displayed as histograms like this one there are different shapes of uh possible but uh a good guess is to use a normal distribution um to apply to our interval there are many reasons for that I spare you that and another distribution uh can be used very well for for example cyber

security risks this is a so-called lock normal distribution and this it has a speciality uh that's the long tail on um some cyber security risks uh may not cost are not not very costly in most cases but in rare cases they can lead to catastrophic outcomes so we can apply this distribution when we think it matches reality better so why doing that we want to change now

we want to estimate impact from negligible to catastrophic and estimating a 90% uh CI for impact in Euro instead so what to do with that now modern statistics and steam locomotiv have have in common that the modern statistics was new when steam locomotives were new too so what did the statisticians at that time did not have right computers we have computers if you do not need to

uh make cers some experiments with uh dices or earns we can just run uh probability experiments with our computer and that are Monte Carlo simulations it's just trying instead of calculating no special software required I sometimes hear yeah we need a special software to do this complicated Monte Carlo simulations no you don't you can do this with Excel if you're not a programmer if you're a programmer

you can use plain python or Jupiter notebooks and uh create in a few lines of code uh uh a Monte Carlo simulation that can generate the data for you so one thou 100,000 or more experiments per seconds that's amazing let's use it so how we bring these together we have the confidence interval and draw a an impact uh for every year in the next 100,000 years and

uh we have a probability for the event uh we can easy create a draw uh with a simple function as well and do this 100,000 times collect the data and then we are able to just count the impact through the different years and uh calculate the probability for a specific impact so that looks like this in practice for example you just have an Excel sheet with your

estimations and probabilities uh and uh event descriptions and when you do a Monte Carlo you can create a loss exceedence curve looks something like that on the x-axis you have the impact and uh on the y- axis you have the probability for the impact so you don't need to look at every event you just approach your boss with this risk assessment and ask him or her will

you accept that is this acceptable for you that we lose uh €50,000 EO with a likelihood of 20% in year he may answer yes or not but uh you're on the on the safe side here so the change we can make here again is show probability instead of a risk Mar Trix was yesterday now we create a loss exceedence curve with Monte Carlo simulations so what what

then what's the next step uh I tell you something about pink elephants that's the other animal headline two persons walking through the forest and uh one is always stamping with the foot on the ground and the other person ask why you do you do why do you uh do you st on the ground and he said I want to shy away the pink the other person said

there are no pink elephants here so why are you doing that and the other person replied so you see works what we want to do is we want to find effective risk mitigation measures we do not want to shy away uh pink can think about risk mitigation in advance before we apply them do the same Monte Caro and create a curve approach our boss again are you

are you willing to accept this and uh you need to consider the cost of the risk mitigation or risk mitigation uh cost some money some time some some anything some resources and when you're ready when you found uh a compromise which risk you are willing to accept then you just apply the mitigations so now let's get back to feedback uh how do we get feedback for our

forast for our estimations of the probability that an event will happen or not and this problem can be solved with the so-called Brier score Glen W Brier proposed the score in the 1950s to uh evaluate uh weather for casts it's very simple you apply this formula uh P of T is the probability T is uh one if the event happens and zero if the event does not

happen and you get a uh score between one and zero where zero is top uh uh flop is one so you need to be close to zero to know that your estimations or your uh forecasts are pretty good and they will not be at the beginning that's for sure but you can get better by just training so there's a project uh the URL is good judgment. comom

that was initiated by IRA maybe you heard of daa the defense Advanced research projects agency IA is an organization uh which is the intelligence Advanced research project activities and uh in this context a project was started that's a good judgment project where normal people were tested how good they can forecast world events based on publicly available data and it turned out that uh some are very good

at that and these people were called super forecasters and uh they were better than the data analysts in intelligence agencies which were trained and had much more data uh and that of course raised attention so they tried to figure out what made these people special do they have some extra sense or uh is this an xfiles case or something like that but no it's not it turned

out uh they are just applying some practices rigor rigorously and uh regularly like finding and using reference class finding the right information they were uh information junkies they wanted to find every information all every time um they find new information they adjust estimations accordingly so the risk management is nothing you do at the beginning of the project and then forget it you have to regularly update your

estimations you can use models also statistic models to understand the uh relationship between different variables uh you can use postmortem analysis to uh get information from past events to get better at future events and of course ask the right question do not simplify okay sounds simple it's a little bit more complicated in practice but it works that's it um I have some takeaways for you and uh

if you like it uh or if you just want to stay in contact and you are interested in other topics about measurement and uh using models to different fields in the it then uh I would be happy if you use this uh QR code for uh connecting with me on LinkedIn and uh yeah it starts with the uh don't be afraid there's no huge risk to do

a risk assessment with quantitative methods it maybe takes takes you a few hours to get into the python coding or how to use Excel for that there are YouTube videos um it's not very very complicated just take the first step and try it ask the right questions that's the most important thing for me start with what you have do not wait for the uh AI solution that

will solve all our problems in Risk Management of course you can ask UI uh chbt or any other uh what are the risk for XY Z that works pretty well by the way uh create models and show them to others communicate about what you think may happen and get feedback create a probabilistic culture what's that create um probabilistic culture is maybe uh in in the world of

poker players uh anduk described that in their book uh when somebody um expresses a claim there's very likely somebody who asks are you willing to bet and that changes something in your mind even if you're very confident if you're faced with the question to bet your own money on your claim then you will probably rethink it more rationally I wish you good luck anyway because luck is

very important to be successful you can make the best decisions and if you're unlucky success come so thank you very much for your attention any questions thank you Mike uh yes you say you say ask the right question so let's go to that to that part uh first of all uh we will move of course to slido questions we have several of those but uh maybe we

have just live questions from audience uh do you have any questions you want to raise your hand and ask something directly to Mike no no okay uh I have a gift for you thank you from organizers I'm not sure it's a giant giraffe or pink elephant one of those definitely and um okay two questions and it goes like that how does a Gile perform in other signin

framework quadrant areas I hope it's it's real question yeah yeah it's it terms yeah okay yeah it's uh it's related to the Canan framework I uh um introduced and uh yeah it's an absolutely valid question um and of course you can apply agile to uh simple uh habitat or to the uh complicated habitat but usually uh it brings more um process overload to these kind of problems

so if you have a a a problem a project you did uh already uh in the same way uh many times and just repeat it and use the methods you used before so uh it brings no additional benefit but it does not harm I would say and one more question uh you described that PMS according to the chaos report lower success probabilities what kind of projects were

in the survey how big were they uh they we of different sizes um I don't know the uh distribution I recommend to look up the C report um it is uh just Google it and you find the results and you find uh much data about uh the different projects they analyzed and it were software project chaos report stands for comprehensive human appraisal of uh originating software uh

that means this the uh how do humans uh uh perform when it comes to the complex task of creating uh software and uh in that context uh I would say if you really want to understand what happened there or where this data is coming from look up the reports and dig into it so thank you so much that's it and Mike Danish