Grzegorz Sztandera: How did I Help to Save the Lives of More Than 1.5 Million People?

In this talk, Jos shares his experience in building a successful operations team that has saved over 1.5 million lives through the implementation of the Emergency Call (eCall) system in vehicles. He begins by explaining how the automatic data transfer system in eCall functions, detailing its requirement for new vehicles in the European Union. Jos discusses his approach in forming a DevOps competence center from scratch, emphasizing the importance of hiring team members with the right mindset over technical expertise. He highlights the success achieved through proactive monitoring, effective team integration, and continuous training, which contributed to a high service level agreement (SLA) of 99.5% in incident resolution. The session concludes with Jos stressing the importance of documentation, knowledge sharing, and team collaboration in achieving operational success.

[Music] launch break over and hopefully you are full and ready to now enrich your brain instead uh just a quick reminder to people as well make sure that you are definitely rating these sessions at the end to know whether they resonate with you where we like them and on top of that remember Q&A is there take advantage of it I really want to see you guys engaging

I know your food ComEd right now it's a little bit of a stretch to reach over that keyboard but it definitely goes a long way and I know the speakers enjoy the back and forth engagement to get you guys what you really want now our next speaker is here to to do a little bit of brag about his Support over the years and how he's helped a

lot of people around the world um they're here to talk about Operational Support it's my pleasure to introduce Jos who's going to take you through a few important lessons hi hello good morning okay so let's start uh warm welcome uh I would like to share my case study of building the operations team that I was uh leading for the last four years and uh during that time

my team and me we were able to save the the lives of more than 1.5 million people uh how you will discover during this session first warm welcome from W in Poland uh we got a great weather today and if you if you will uh visit Poland and especially W in the future please uh contact me I will guide you for the best places here okay uh

some more about me you can scan the QR code and connect me on Linkin uh for a long time I'm taking care of projects and products in in different areas like banking automotive Telecommunications uh over five years I spent about the devops and service management and personally I'm like extraordinary uh travels so for example on the picture you can see me in front of the fourth erao

of the nuclear plant in Chernobyl uh I'm of two uh I like painting the B figes and also I wrote a stories for kids unfortunately in Polish so uh maybe in the future there will be some English translation a short disclaimer uh I am working for multiple companies in my history so what I will say to you today this is only my opinion and does not reflect

to any companies that I'm working where working or will be working uh in the future and also uh every one of us has uh potentially joined multiple projects every project is different so uh please have in mind that my perspective may not reflect what exactly you have uh experienced in your uh personal uh history of of leading and joining the project on the it or other areas

okay so let's start our pipeline for today is uh first I would like to introduce what is Eco and how we are able to help saving lives then I will share with you my plug-in approach approach for building the operations team that I will expand uh in the few more slides uh then I will share some statistics uh from the project that my team was taking care

of and also I will uh bring few projects on front just to give you some summary what projects uh my team was taking care of and uh before the summary there is WTF maybe you can uh discover what this abbreviation is for so in this uh presentation it will be white technical failures so in case that we'll be able to cover this within the timeline that we

have have here I want to share with you few of the technical failures that my team during the last four years uh has discovered and was uh was HED on production by the different circumstances also uh after my talk there will be a panel discussion that I will join with with some other panelists and we will talk more about the technical failures on the devops area so

please be invited to join also the panel discussion later okay so let's start uh I don't know uh how many of you knows what Eco is but generally it is the emergency system that in case of any accident or other situation based on in vehicle sensors is automatically transferring the data uh to the to the public safety uning points and uh they can send some help to

rescue your life this is mandatory for the new vehicles in European Union if the vehicle was um granted the access to be sold on the market starting from 2018 every single vehicle should have such system on the left picture you can see just sample uh picture from one of the car manufacturers how the button looks like and on the right when you will just press it it

asks you to hold it for three seconds to call help because it is working in dual mode one you can just press it and ask for help because there might be a situation that the uh in vehicle sensors will not uh see that there is something that you need assistance and the second is that uh generally uh the car will call the help automatically just on the

short uh picture how it is uh going to to be so you can also notice that starting from 2018 every single car has the GPS system inside because this is needed to cover the uh Eco requirements and uh you can with the Eco system send the data from the car just to receive it from the pup and then they can uh order some help for you or

you can also talk with the person from the pup it can be 112 uh emergency uh Co line and uh you can just simply say what is going on how much H or or which kind of Hub un need this is the eal that is currently uh working and my team uh was providing devop services for the product that was a part of the eal process uh

it was not covering all of the cars in the world but just part of them and during two years from the statistics uh the application processed over 1.5 million eal so my team which was responsible for providing the devops environments on production and multiple other things together with the 247 support we are responsible for having this product live working and being able to process all of those

uh eals every one hour for example during this session it uh if the application will not be working it is potentially around 100 eals missed so so if the application is not working for in one hour uh we may have an issue of not being able to save livs in 100 cases on the roads so this was really important to be able to cover this 24/7 and

the reaction time was really really short for us so I don't know from what country you are but generally as per statistics during this talk five such eals from these systems will be raised in your specific ific country okay let's move forward so I joined the company and were asked to deliver the devop competence Center devops Services together with 24/7 support So at the beginning I had

three not devops Engineers but more I will say application uh operations people that were able to discover the logs and and take care of the uh process of handling the incident not resolving the incident by themselves at the beginning and in the first uh two first month uh two of them uh give the resignation letter so in sometime I was alone with one person one engineer uh

covering 247 working of this specific in systems so it was really a tough time for me and I really needed to build the team really from the scratch so as you can see during the first two years we come to the we came up to to 10 engineers in in the team and uh due to lack of competences at the beginning people and resources uh there was

overall number of over 1,000 overtime hours just to cover the needs to deliver the best outcomes the best quality and UND distrupt uh process in example of the Eco so maybe you wonder how did I uh start building this team because at the beginning that I said there was no competencies about devop about for example AWS uh how to set up the cloud environments and so on

so everything was going to be uh discovered during the phase of building the the whole competence area so uh I asked people to join the company and uh we had the people with maybe not the senior uh the senior level of the devops engineers because I know that after some time they will be a bit bored about the cases and solutions so uh I prefer to to

hire regular or Juniors uh with some strong knowledge or strong willing to to discover the knowledge in one specific area so for example uh on a daily basis uh everyone was able to perform the standardized tasks that we had in the team for multiple project but I've got the Specialists or expert uh in the databases in it processes to to to cover the best uh outcomes from

the support perspective and also awcd and other important factors that were uh for the projects really critical to handle both the devops and support s in one time I have a support of line manager and uh this was also important that I was not directly responsible for the salaries for the people so it was easier for me to cover uh the things of building the relationship between

us so at the beginning it was really hard and if you are going to set up the devops uh competences services within your company uh I will recommend not to hire just five devop seniors because uh if you don't know what will be the flow of the work for for example next one or two years uh you may have a situation that they will be bought and

they will quit and escape from the company it is better to build everything from the very beginning and have in mind that the people uh are able to learn from each other my personal believ is that if we are a team if we are a group of people that have the same uh Target and we believe that we can handle it we may even not have the

proper knowledge because we will be able to discover it and resolve the issue very fast if we will be a group of seniors of expert but individuals and not sharing our knowledge we may have a situation that someone will be off and we will be totally not able to resolve any issue on production okay so the left on the left you can see how many deployments we

have during the time so as I said at the beginning I was building the team but after a few months it was really a critical time with all of the with a lot of deployments on production uh were a bit breaking the develops rule because deployment was in fact released because there was no possibility at this stage to uh have it not in parallel uh how the

team was work uh I prepared the on boarding for every single new uh member of the team and after some time I discovered that uh I am forgotting about something when I'm uning a new person so I uh followed the rule that the new employee was on boarded by the last hired employee and it was working really great because uh the person that was the newest one

in the team was has the fresh knowledge what exactly is needed uh to discover and to to gain the knowledge at the beginning for the specific projects that the person is working with uh the second point is the boot camp program so after one two years together with my my team we build the bootan program for devops so it was quite an unique unique situation for the

company that we were hiring the junior devops Engineers sometimes even out of the it market and and out of the it area and during the three months we were teaching them with the specific boot camp tasks uh how to be a devop what to do uh in the projects we teaching them about the cloud concept about cicd processes about the support and so on it was divided

into uh two tasks first was I will say self-learning uh uh and the second one was uh more based on on scrum methodology proving with within the springs that we are capable of of delivering what we learned at the beginning of the boot camp program and if you are looking for building the competences without uh within your company and uh you have just few devops Engineers you

want to extend it uh without hiring the new seniors and having the high cost at the beginning I recommend to build a boot camp program because it was very effective for us uh also is important with being the team I believe that personality as I said before is more important than than the tech skills because you can learn the tech skills but you are less able to

change the specific person personality of course I delivered to the team all of the necessary training and certifications regarding AWS it and so on uh and the other case for building the team was how to integrate them so um I will say a bit more about it on the next slide so at the end the team uh feels the responsibility about the projects that we are running

both on the devops and support side and we trusted each other yes uh so I will bring one situation during the weekend uh we had of course the 247 support uh ongoing for projects and only one person was called in case of critic situation everyone else just received the standard SMS system SMS message from the notification system that something may be wrong and generally didn't have to

didn't have to uh perform any action but I had some I had some time so after 15 minutes of receiving after receiving that uh SMS I joined the call just to see what exactly is is going on uh how the person that was on duty uh is is taking care of the issue and generally I found that that the whole team every single member of my team

was on that call because everyone has time uh everyone wanted to have and it was really showing that we have we we had the uh common understanding uh what is going on and also we were a team so it was not the most important who was responsible for uh res resolving the issue that time but if someone has time he was also able to join and help

on the management side of course there was a lot of risk that I had to manage I had the separated uh jira uh for for my risk and I was managing it on a at least weekly basis okay how to manage the team also uh to be visible in the company so as you can see H I prefer to do the team identification we got the t-shirts

we got some other brandings that uh the whole uh company was able to discover that yes uh we are devops there is something new in the company new competencies we can I will say use those competencies also in other projects in other business areas so I promoted the the competencies of devops and support uh across the company and it brings us with more and more projects at

the end uh personally I had to learn multitasking I will show you how tasking was uh in my calendar at the next slide uh but also there was a lot about internal development not only myself because before I was not aware of I was not a devops engineer yes so I had to in a short time gather the knowledge about all of the cloud cicd and so

on Concepts but also for the whole team including as I said before also external trainings and development very important was to document and standardize what we know because we were working with multiple customers and multiple Pro projects so to set up and document specific processes and to gather and gain knowledge from everyone important after that I started also uh to lead uh discussions with the existing and

new customers about acquisition of new Services negotiations and transition of the services to my team uh there is a organization in Poland that is publishing results from the it managers uh survey and there are three exactly four uh points that you can choose when you are describing yourself as an IT manager you can be people leader uh business leader technical leader or uh organization architect so uh

I will recommend first for building the team and and being the people leader because um if we are a team when we are a team we can do everything when we are group of individuals it won't be so effective as it could be okay let's move forward so now you can see one week of my calendar from the pre from the previous uh company when I was

building the devops competence area h i highlighted six meetings in parallel so uh I go as a manager to a point that I was not able to do anything because I was just jumping through the meetings and bringing a lot of cases and a lot of action points from one meeting without having any time to uh do it yes so I had also to change uh my

way of working uh to give more of the uh responsibility to my team uh and funny thing was that at the end I was able to join two calls in parallel one on my laptop one on my mobile and I was able to listen all of them the the case was if someone asked me for something it was really hard to know which call uh they are

calling me to join the conversation so uh during those years uh from starting from one project one customers at the end there was seven customers and 12 projects running so uh it was really I I'm not able to talk about the numbers but the revenue growth from the first project within my one year was uh very higher than at the beginning of of my uh start of

working at that company and after two years it was really uh uh it was really higher uh multiple times the revenue from the this area of of competent Center of devops and support services so that was the business effect effect that uh were built during the building the team and the team spirit within okay statistics from one of the projects so uh we were handling in the

application over 200,000 processes weekly and during the whole project we did over 500 deployments also what is important that team uh was very proactive so we independently from the developers testers and users we identified over 400 defects by our own uh also in the code because we are very actively reviewing on a weekly basis uh the locks uh from the monitoring systems just to discover the coron

cases that were not able to be catched by testers or were not visible for end users on production during that time we also received over 1,000 incidents and our SLA uh to the resolution on time was 99.5% what is quite interesting we where the application was connected with multiple other applications within that company delivered by different uh from different sources and by different companies and we also

had a need during that time to raise over 600 incidents to that other apps and unfortunately we were not receiving the same SL level as we were delivering for the product also we developed independent monitoring of applications so just just just just long story short there was a standard monitoring systems uh that were delivered by the company that we were providing the application for but also we

discovered our own monitoring system based on even simple nagio uh that the system was able to uh inform us about potential failure within few minutes while uh the customers monitoring system was alerting us after over 15 minutes from the critical situation happens on production and this also brings us to the possibility of being able to deliver so high SLA during the whole project okay just a few

words about the project before I will show you how at the end the team uh was working and what was the results from from beinging building this whole devops uh service competencies first the automotive one we were responsible for taking care of the application that was uh taking care of user requests yes so for this project we are delivering the whole cicd process including deployments and frees

on production we were taking care of the support 247 everything was on AWS covered by multiple monitoring systems um I will not name the the the exact systems now that we are using but overall for the all projects uh within the team uh we are using K grafana din promus data do and probably few others that I don't remember right now uh and we believe that if

we can do the monitoring we should do it right and if we can monitor something from uh two different uh systems or sites we should do that just to be sure that everything will be working smooth uh on on production as I said also proactivity so log analysis analysis and after some time Automation and scripting just to re just to be able to do something else during

the time that we were providing something manually before next one was for the aviation so uh we took care about critical process because ground aircraft communication if you will be flying uh through the Europe for example uh uh for your vacation time uh you for sure don't want that the plane will lose the communication with the ground systems so here uh we provided development in the in

the standard scrum Ed approach uh on our side as a devops and operations team was 24/7 maintenance with monitoring clock analysis and also Incident Management the last project that I was to want to highlight here is the health industry uh which got the covid situation in the past this is not connected to with with covid but uh we provided the full devops for the solution that was

taking care of the drug testing including uh AI uh and here from the very beginning we provided uh the first environments in the cloud for the customer uh we provided the first uh optimization for that including after sometime uh automation uh with the infrastructure as a code for building the new environments automatically and at the end when there was some time we also optimize the cost of

the cloud concept at the end because the customer has multiple environments consuming a lot of uh a lot of uh resources okay how the plug-in approach Works uh at the end so basically as we have a project we had a project with the development we have some Developers testers management so the whole I will say scrum area from the second side we got the customer uh it

can be an end user or company that we are delivering the software to and we are connected with Aden other vendors and thir part companies uh within multiple of of Integrations and my team uh the operations team took care of the providing the environment monitoring for it all of the it processes uh within 24/7 Manor the whole delivery of the solution and production including hyperare after deployment

and as I said it was very practive and at the end a lot of automation was introduced just to make our work easier you can also divide the plug-in approach into three three IRAs so first is the death so we are building the environments or of the or or the pipelines uh or creating some python or or or bash scripts so I'm calling on the operation level

of course that this is something like service architecture then we have the operations so we are running what we have prepared both about the environments and the pipelines we are providing the monitoring it with the proactivity and hyperare and also building the knowledge base which is very important usually in the project there is no time for providing the documentation and uh this is okay maybe for developers

because they are not uh working 24/7 to maintain the product up and running on production but at the end this is critical to have the good knowledge base so this is the part that I'm calling the service management and the third is the 3PO so observability optimization and operations so we are supporting the solution that we bring to the production uh within 247 and we take care

of everything that may happen there next Approach at the beginning I said I was building the competences within the team so at the end we were certified from starting from it through AWS Oracle cloud and Microsoft uh we took part in the regular Sprint bulock tasks related of course to to our Dev and support area within the project so we are we were not only taking care

of walking environments on production but also we received the task during the Springs to deliver something new for the whole application uh full responsibility for the E2 and pro uh environments uh and helping also uh the team of developers and tester with their uh int and test uh environments so so lower stages uh as I said we are setting up everything from scratch we were able to

deliver it uh release deploy and release the application and also take care of the whole production environment this is the standard uh devops loop so if we are looking on it uh we are not able to code and to test because we didn't have the competencies of a testers to prepare for example uh automation test and we are not able to bring the code because uh our

competences in case of uh writing a code was on the junior level just to understand what is inside the code okay now I would like like to jump I'm I'm just checking how much time we have and we have few minutes so I want to share with you some of the situations that hited us on production during that few years of of building and uh leading that

theop steam uh not always it will be due to our issue and if we are working in a complex environment it is usually not our uh fault at the end that something is not working so uh I will talk about the application that has a lot of endpoints about 20 or 30 uh Integrations with other applications around and uh first is missing the s for the tick

and it is the human fault of us uh so not always the really I will say sophisticated uh situations on Productions brings us to to to miss the SLA uh here we bridged it uh because uh we were in the middle of fixing the production and the monitoring system was also automatically creating the incidents to us and uh it was more important for us to fix the

production than to take care of the incidents and unfortunately we just uh forgot to close the incident on time but the production was up and running so uh it is not always easy to uh deliver bet best outcomes you can do everything what you uh can to bring the production up and running but you can forgot about something that is in the process just to Simply close

the incident the next case will show the external responsibility that for example uh we got an issue that uh users were unable to log into the application it was critical uh incident so we had one hour for resolution of that and uh at the beginning in in our application when someone was starting our application he was redirected to external two fact authentification system that was authorizing this

user and then uh the flow goes again to us with the proper authorization and we discovered that this is not the issue uh on our side and the users were not able to loog in because the 2fa uh was not working yes they of course has some unplanned outage uh they Implement some uh new functionality or the change request on production and they didn't in inform us

that something may be going on uh but we were prepared for that because we are starting with our application without two-factor authentification so we know what we should change to disable toctor authentification of course we can discuss what is more important uh to Factor artification on on on login page or having the application up and running for some time uh here the application was so critical that

it was more important to be able to log into application and use it so we were during the deployment on the E2 environment with uh the changes that enabled uh us to log in without enable users to loog in without two Factor authentication uh and during that time the problem was resolved by the by the uh 1218 so you should during building your application and your devops

area and environments think what can go wrong here we were luckily uh aware of of this and we are able to change it by hour so to without any other steam So within two hours we were able to fix not our fault and bring the application up and running okay let's move forward sometimes backup is is not enough um so the application was not accessible for all

of the sensors that were using it uh and the question was on the network side yes because the primary firewall that was not in our responsibility to take take care because there was separated Network team taking care of all the network stuff uh the primary fire firewall went down and uh maybe you will not believe the second one the fail over the backup one also failed so

you are out of the network because there was no firewalls and they needed to do some manual failovers and resolve it so we are just waiting for that but you may consider in your project if you really have the backup and if the backup is working for example uh the other situation not listed here uh we had the I will say standard server uh Data Center and

there was some systems that we were checking and when we were checking the first system uh it goes down and automatically the second also goes down and it was uh a problem on the hardware level because the firewalls were were not working and luckily we had the third one uh available so though this was the third one was our real backup because those two that were working

the first primary and the second backup was just the configuration for the production so always think what you can have as a real backup in case that the primary and secondary Solutions will fail and it is not always uh good to be the first in one of the projects we are not able to log into the application because our configuration on iws was deleted why because we

have fast in the cloud the customer started his Cloud journey and at the beginning there was no procedures processes um I will say dedicated team responsible for taking care of the whole Cloud environment so we were just able to deliver our application deploy uh the environments uh on AWS and after some time we discovered that we are not able to look into the users are not able

to look into the application the case was that uh the team that finally was responsible for uh environment uh introduced um introduced terraform scripts just to have uh all of the environments on the same level with with the common settings and they just forgot or don't ask or missed that we were before introducing the process of gathering those configs and they just deleted it totally so it

tookes a lot of time for bringing it on production uh and next day they run again the same terraform script and we were HED by this situation again because they didn't introduce the changes uh to the scripts uh so this is all of the situation that you may be ready with everything but unfortunately due to some external reasons you may have a problem on production okay L

forj around two years ago zero day exploit was identified and a lot of us has an issue with that so we were uh aware of that we go step by step to upgrade lock forj together with CDL uh of course we are we were unable to bring it to the life and environment within one hour because we have four environments we needed to go one by one

deploy it we need to at least perform some simple test at the beginning uh and at the end the operation was successful uh we were not able to uh shut down the environments at least the prod environment during that change and the next day we were informed that potentially were attacked 16 minutes after receiving the uh need for for for upgrading it so there was really no

time for for for doing anything what we did we did the local dump of the database just from the current time to be able to see what was potentially CH checked for the for the security analysis and also uh we bring deployed the application from the very beginning we uh bring the database from several days before restoration and uh analyze the traffic to the environment because it

was only to e so in the future uh as we are unable to uh shut down the production we know that at least all the others uh environments we can shut down at least if we do not need them for deployment of the final solution to production so uh doing everything what we can uh we missed something that we could do and uh this is something that

will be our lesson for the future and the last one that I want to highlight here we received a critical incident that application is totally down we were checking at the beginning as always if everything is working for us so working for us yes works works for me but uh as this was an important application we wanted to have some more information from the reporter we tried

to reach him uh we tried by email phone no answer for the two hours after that he called that he was at the market to buy some vet tables and uh at the end this simple thing consume the time of three or four devops Engineers out of ours trying to resolve the issue that does does not exist because in fact he said that it was just three

minutes long Network issue so as you can see during my slides I'm always uh putting the M of it is fine so this is not fine okay summarizing this part before the issue production you can prepare the documentation processes procedures have the proper training and share the knowledge when you have the situation on production you can identify what exactly is going on integrate the resources and competences

that you need for it for resolving it intercept and inform all of the people around because maybe someone will know something about this specific situation and investigate this issue till the resolution what is also important after you should go the knowledge perform the retrospection just to know what exactly was done what we could do better fix the issue if that was just the hot fix during the

situation and document exactly what was done even for The Simple Solution even for something that you think will not occur again I had in my history the situations that were HED after two years about the same situation and if we will not have it documented on the Confluence page we will probably spend few hours for resolving because no one uh no one remembered exactly how to uh

take care of such such situation update your processes and processes okay takeaways when you are building the team when you want to build the devops competence Center remember that personality at the beginning is even more important than uh technical level and in the long term it will bring really AOSS uh to you small teams of course are more flexible and if you are beaing the team from

scratch it is easy uh to integrate the team line manager is important but the team leader also can intersect and for example taking out from the team leader the uh part of the salaries will help you to integrate the team you are not onean Army but one team army and uh together you are strong individuals cannot do what the team can uh and also I'm trying to

to deliver to my Engineers the broad context on the business operational and so on uh site just to give them the perception what exactly is going on why we need such actions to be performed and at the end be more people than Tech and Beast okay if you have questions I will be very happy to answer that I hope that I will have answers exactly for your

questions uh and one second thank you very much uh you can join me on LinkedIn uh and we may have a discussion about how to build the devop steam if you will have the devop steam already build it then you can go to the platform engineering and create even something better thank you we are muted thank you there we go I was G to do that um

yeah thank you very much shos um so if you guys do have any questions for him Q&A click the tab type in your questions right now of course you can always reach out there if you prefer um but yeah really quite a thorough discussion there you know first thing that kind of came to my mind I mean it's kind of going to be an opportunity if you

just kind of brag uh what would you say like using these kind of practices what was your biggest I guess win right like what was your would you say is your biggest achievement using what you've learned here like an example where you went in and turned a big disaster around but I gave you the example that the whole team was available during Sun when when the the

production was HED by a critical incident and this is my win because uh I personally I'm not on the Deep technical level that I can create the whole iws environments and so on but uh that show that I can uh deliver and and build the very effective teams and that was really important for me yes because as I said I believe that if we got the team

we got the Fate we we are feeling what we are doing we can do everything everything if it will be just individuals nothing will goes maybe for some time we will be best but at the end there will not be the same Spirit the same effect and so on so for personally for me this is the biggest achievement and the second achievement is that I was able

within few years to build 10 people uh to be the senior devops Engineers with multiple clouds multiple cicd Solutions and so on understanding the whole concept being able to work in par on on on two three or four projects that were running with different uh different uh coding language different cloud different cicd solutions and so on and so on do you think um because you mentioned in

the first part that you mentioned you're not the most technical person the second part you highlighted all these different platforms do you think that's kind of U an advantage for someone in more of kind of that managerial position is the fact that sometimes not being as deeply versed in the tools gives you a a clear perspective on what to do from a a high up um so

position at the beginning of my journey in the in the dev OB Seria I uh took my personal time and I made some some certifications uh on the cloud level at least uh but I stopped at the level at the beginning when I was with only one engineer I was even doing some some some work and tasks in in AWS but after that there was more work

for me not on the technical layer um and I think that the good approach is to understand what team is telling you but not be exactly being able to do that yes but understand that this is the good approach understand that uh they really uh think about what the solution will be best for this situation and you can see that they are using what they should yes

yeah so understand sorry go ahead so understand not not perform yes yeah I I I guess like the interesting way of thinking that is you don't become too romanticized with a specific solution you can see everything with a more unbiased lens right because you're just looking for the optimal end solution instead of going oh man I've used this for years and I love the way it works

yes also some of the people has the approach that they are willing to do something by themselves yes so if you cannot you you don't have this problem yes yeah um I think that's all we've got time for we're running over a little bit shagos so I'm being conscious of that definitely a pleasure to talk to you uh people have got your links they want to reach

out to you but thank you very much for a really informative talk thank you have a nice day bye goodbye