Yishai Beeri: DevOps 2.0 - Bigger, Badder with More Automation

This talk covers the evolution of DevOps practices, highlighting challenges in software development and continuous integration and delivery (CI/CD). The speaker discusses the increasing role of bots in managing code processes, including dependency management and AI-assisted code generation. They explore the bottlenecks developers face, particularly delays in pull request (PR) reviews, and propose solutions to streamline these workflows. By introducing measures to automate reviews based on PR complexity, enhance visibility, and improve knowledge sharing among developers, the talk emphasizes the importance of adapting to new technologies while maintaining effective collaboration in development teams.

[Music] ladies and Gentlemen please welcome our next speaker yesai beri presenting the topic devop 2.0 bigger better with more automation thank you everyone great being here um especially excited to beus for the first time like like you mentioned I have uh I have Roots here um so 100 years ago my grandfather started his life he was about uh 20 years old started um life and you know

what's the best vocation uh we were all in the space it was technology so he went to study at the or technicum uh here in in downtown uh the old town of vus technology has changed a lot in the hund last 100 years um I looked at the list of subjects that they learned in the school uh some names are familiar a lot A lot has changed

um so fast forward and we're all doing devops we're writing code we're using uh pull requests um and I'm going to talk about the next things that are coming in the space the next challenges facing all of us as developers uh pushing code to production and continuous merge as as a suggested approach to solving and addressing some of these problems a little about myself um I'm from

Israel from t VI I'm the CTO of startup called ler B uh operating obviously in the space um and happy to to to be here I'm going to talk about cicd and some gaps and some challenges that are starting to or have already started and are um growing fast in this world I'm going to dive a little into uh data and research that uh we have to

show about the Dynamics of what it takes to get code all the way to production I'm going to talk about Bots and I'm going to start thinking step by step into what a solution could look like how do we address uh and what are the building blocks that we need to address challenges so I'm starting with a brief history of software development and delivery way back when

I was beginning my career we had the waterfall um prds and and giant gun charts that you would put on the on the wall and they would get outdated the minute you printed them and You released every 9 months every two years depending on uh how you worked some organizations still live in this uh world but the majority of the world has moved forward in the '90s

in the uh 2000s we started doing agile and now we have Sprints and story points and planning and stand-ups and a lot of um I would say religious debate about what is agile how to do it right why is it not working for me or is it working for me and so on um and then devops a little newer newer another uh name which is trying to

bridge what happens between the code and the runtime am I a developer and I'm an Ops person is there a difference um all of that but now something is is changing and I'm going to this is like a teaser I'm going to show more data in a bit so the Bots are coming then there's two types of Bots uh two rough classes in this space and I'll

talk about why this is interesting we are starting with a stupid Bots anyone here using dependabot in their organization renovate something to manage dependencies I see quite some hands these are examples of pretty simple rule-based Bots typically for compliance or security or other reasons and their job is to create a pull request for you with let's bump your a library version of something you're using because it's

it's no longer the latest there's has of vulnerability it's a nice automated way to to keep my codebase up to date they are creating PRS they are creating a lot of pull requests someone has to deal with them so that's already starting and we'll also show some numbers in a bit that's to shift some Dynamics in how code and what code is getting created and pushed how

systems are changing um and then then just around the corner there's the smart Bots anyone using co-pilot or similar uh technology to write code to write tests so the focus so far has been mostly on helping me in the IDE and but we're already seeing multiple vendors and co-pilot not lagging much behind creating bots that actually write code write pull requests like create code put up a

PR and uh sometimes help you review PR right T so it's not no longer an assistant for me as a developer it's becoming even more autonomous so let's par this thought for a minute but remember that the Bots are cicd like a very um simplified view of you know what happens as developers work and together code all the way to production I write my code I fix

the bug I you know write the feature then I create a pull request who is using pull requests in their day-to-day job here everyone who prefers to call them requests okay I get lab people nice uh I create a PR sometimes that begins some CI jobs some tests are beginning to run maybe built um all kind of assisted ways to let me know that the code is

okay the pr gets merged after some debate and then there's an integration test additional work and then you know CD hopefully or some version who has a fully blown CD automatic deploy once you merge to production yeah this is very few probably five or six people here this is a promise that is yet to materialize but it's the Holy Grail right code is great it's it's it's

merged now it's in production people are getting there but even if it's not completely automated someone has to push a button there's a lot of automation about making that move no one I hope is ftping to servers to put the binary there I used to do that a lot in my in the old times uh so methods to push the code all the way to production you

know with containers a lot of work in the devops world is about the part the last part deploying and then managing the runtime I'm going to focus on developers not on the so when when you when we look at the actual patterns what happens is in this step between pull request and merge code is getting stuck when I write my code yeah I'm using co-pilot it helps

me I'm looking I have this big control c contrl v keyboard with stack Overflow everything is nice that's me and I'm not locked hopefully then I put up a PR and now I have to wait so if you look at how we split the time what we call Cycle time which is a measurement of how long does it take me or the organization ation to take a

task from beginning to end first part is coding that's me working if if I know what I'm doing and I'm not to overloaded and I don't have too many boarding meetings I can get you know get this going fast pickup is how long does it take for someone to take my pull request and begin to review it it's basically waiting in the queue for someone then review

is the time that it takes for all the comments question questions back and forth all the way until we agree and get this uh this piece of code merged so there's a lot of back and forth at least two people involved maybe more and it takes days in many cases and finally there's the deployment how long between when the code is merged and all the way to

production obviously this depends on your CD practices your what is your production like and so on this is typically out of the hands of developers so we can invest in CD to make it happen so sooner but we want to know how long my code is waiting it's already done it's in in in the main branch but it's not yet released uh the main bottleneck is in

the middle is this familiar does anyone here feel similar patterns when they're working on code waiting for get it to get a review and waiting for the review cycle to finish yeah at least some people here so this is and that's what we're focusing on that's what the data shows us there is a lot of empty or or weight time happening here and this means contact switching

if I'm waiting for something I have to get back to it in two days after someone gave me a review this is an expensive context switch I already forgot what I was doing here I'm busy with with my next uh feature um so this back and forth is very expensive and this is where developers are getting frustrated you ask a developer do you like to produce to

uh you know have your code in production to produce value do you like to merge they say yes so dope I'm in rush to to get a to merge my PR frustration is it works for me there's a CI flaky test you know that and then uh waiting for the review okay everything is ready I'm done why is it taking three days to get some lgtm comment

for someone a lot of frustration a lot of dead time a lot of contact switch um and things are not getting better I want to double click in some into some research that we are we a lot of data visible to us because we connect with our customers um get providers and you know fetch all the metadata uh we have tens of thousands of teams on our

free offering so we have a lot of data coming in and now we can start understanding what are the patterns is this really a problem when does it happen and so on so one very obvious thing that we see is that pull requests you look at their life cycle when PR was created all the way until it's merged most of the time they're idle nothing is happening

and I'm when I'm saying idle it's not just yeah some half an hour pass by with no change it's it's in no one's mind it's out of focus or out of like swapped out from everyone's mind for hours so you look at this about half of the PRS are Idle No one is looking at them about half of their time like a life cycle a PR is

up for 3 days one and a half days is going to be idle completely no one has touched it in like in hours uh and a third are idle for even more like 80% of their time they're mostly wait they're sitting there and waiting and if there's no movement nothing is getting merged all this great value is not making it to production to help your users to

give you a feedback loop and remember the contact switch for developers what we also see is once there is a one a delay has begun you're you're uh already slow with dealing with a PR it gets even longer so of the PRS that are spending more than one day half of them are idle for like 80% of their time if if a pull request can be done

you know created and merge in one day that's a huge win and the we see that the more delays are in the beginning of the pr the longer it will take overall so one thing that you know if if you can get into a same same day PR that's going to save you and your team so much trouble and so much time and uh frustration remember the

bots so now we have some data for Dev orgs that are using a dependency management bot like renovate or dependabot or snake or others almost 14% 13.3% of the PRS were that they actually merged are created by the bot so it's not like a side thing this is you know and we have um we're seeing customers typically on the larger U and Enterprise customers with 20 25%

of the PRS created by bots so of everything we do if everything we review and merged 10 to 20% is like is not even human created and this is just the beginning um we think about the co-pilots and the others it make making it so so much easier to create code uh and you look at the dependabot U PRS about 40% of them are dead simple this

is a patch update for a library like the very very minor version and it's just sitting there waiting for someone to to have attention to take a look in many cases what do you eventually do with these PRS let's let's try right pass the tests are passing I'll lgtm it and see what happens most people don't actually go and do manual testing or review the release notes

for a patch release obviously this differs and not all libraries are the same same but there's a huge amount of work which is basically toil just waiting to get uh and it gets in your way you open a PO request view in GitHub it's all dependabot stuff unless someone is on it this frustration is not going to get better there's like more incoming on the code side

with you know once those llms or um um gen Bots mature which is you know in six months they are starting to create 10x more code more PRS for you you already have a bunch of automated PRS just with the pendot think about all the others this is like an um I don't know what's the best uh metaphor Avalanche of PRS of code that no human created

and has to go through the same the same funnel the same pipeline someone has to review it someone has to say it's okay someone has to merge it and then deploy it so this and the question you should all be thinking about is is our process is is my pipeline not the build pipeline I'm not concerned about builds right spend a few a few more dollars on

the build machines the builds will happen in time the human pipeline the merge decisions the reviews the are these are going to break if I'm not changing something Dr atic because my inputs are going to be different so that's the problem statement we're already seeing the problem even before the Bots and now the Bots make it even even harder and think about you are getting a review

from one of these nice icons a PR uh and you how do you review a pull request that was created by a bot how do you defend a pull request that no one created who's going to reply to your comments who's going to take ownership like this is a code that no one read no one wrote and no one knows how to review so all of these

things are challenges in my pipeline to get my code in good quality all the way to production so now that we've laid out challenge what is missing and we're now is the part where I'm trying to build layer by layer a solution or what do we need what are the building blocks we need for helping the developers and the people that are responsible to get code merged

and deployed help them do it with and maintain their sanity so a few pillars here one we have knowledge um developers to to do a good job in um you know creating their code remember the my PR has to compete with all these bot PRS and all the others and get a good review for me as a developer I need to know who should review this in

many cases in large organizations I it's not always clear who the reviewer should be I can spend two days just hunting for the right person so I'm going to ask someone if they're not the right person they're going to ignore me typically so just finding the right reviewer in itself it's a piece of knowledge that many missing um or maybe the reviewer says yeah I can review

but there's another person or maybe let's find a better one or someone to work with me that's all knowledge which is hard to to find understanding what is the process what do I need to do to get this V emerged every development organization has some different process like a Unwritten lore of what it takes to get a good PR to have it actually reviewed and merged sometimes

you have a list that you need to fill in sometimes oh everyone knows that you should also do this and this but actually no one knows if I'm new in the team no one tells me so learning in the hard way what is even required of me so my code can get merged and what's the best way this is like some soft skill skills I know we

developers are not always great with soft SK skills what what I need to how do I push this forward who do I nag when do I nag do I even nag how do I get this done this is all knowledge that developers can benefit and you know have a easier life if they have this for the reviewers in many cases what they're missing is context I get

this big wall of red and green and GitHub okay why is this code written what is it for what is the change actually doing here uh is it urgent do I do this now or maybe in two hours when I have more time how how long will it take me um what's the safety or risk levels here so all of this is context that can help reviewers

make better decisions even at like do I even start right now uh one thing that we've seen very uh successful in heal helping reviewers do the right thing we have a nice machine learning model that says this review is going to take you 20 minutes it's like a estimated review time like you have with articles on the web says reading time similar with review time and now

you can choose I have do I have like is it a two-minute review I have this now between meetings I can do it or this is actually going to take you 30 or 40 minutes you maybe want to schedule it after lunch so uh just having the context up front in the notification that I get about the pr review the Prest waiting for me understanding how long

is this a big one or a small one a complex one that that helps make make right decisions the next part is about flexibility and Automation and recall the bot problem that I mentioned if I'm automating the inputs system resulting in a 10x or I don't know 50x input size problem and I'm not automating what happen happens next I am bound to fail there's no way we

humans can keep up if we keep doing the same so there's two parts here flexibility is about should all PRS actually go through the very same process all the time like yeah everyone has to review a PR every PR needs a review we all basically use a single process a single threaded process for every PR they all look the same and we should ask ourselves is this

true should all PRS get processed in the same way and I'm saying no so flexibility I'll show some examples uh and automation are there places where I can skip a human review maybe this is a a p request that change a documentation line maybe there's no need for a review a a coder or developer to review this or someone has changed a static asset in a like

your website there's no need for me to to review this it's not code um and there's many more examples can we skip some review sometimes and automate those away patch updates from the penot no one is actually looking at the code right it's just a number in a requirements file or similar so why do I need a human to say lgtm can I remove toil that's all

toil that we can reduce and automate to match the automated inputs on the other side are there places where my uh pull request needs a more complex workflow I need to pull in someone from the security team and make them a required approver for this spr why because there's a security issue here someone is touching the API someone is changing a port many ways in which we

can say this needs more so I can do less on some PRS and I can do more in a structured way on some PRS now I can be smarter about how I review my codes so flexibility and automations and finally visibility if you're managing this problem you need to measure stuff you need to understand how long are we waiting for PRS is there a bottleneck somewhere is

there a single person who's doing all the reviews and now they're away for you know 3 days and everyone's stuck so visibility into bottlenecks visibility into the patterns is another layer that you have to have to manage this problem at scale to know how much to invest there to know how much this is hurting you and so on I mentioned the estimated review time this is a

graph again from our data showing the impact of knowing ERT into pickup so on the x axis you see the pickup time how long did it take for the P request to get the first uh comment the first uh review artifact basically how long did it and sorry Miss on the X x axis is the estimated review time this is the message we gave the reviewers this

is how long it's going to take you bucketed into different so and on the Y is how long they the pr waited in the que and you can see if the pr is small and the the signal is that this is a very quick win for you you can review this in a few minutes the pickup time is very is very short if we told the developer

the reviewers this is going to take you almost an hour then pickup time went you know goes way up so reviewers are responding to that signal and making different decisions uh on where when when to start this is not about completing the review it's when I start the review so just knowing that this is small makes it happen sooner which is a win for everyone okay so

some examples about flexibility and automation that I mentioned so ship show ask is a a nice model uh one way to think about the problem instead of having one um process for PRS this is a framework that says for some changes we trust the developers enough that they just ship it okay that's the first one just merge your code for some cases you ship it and then

you tell everyone so I'm showing you hey everyone notice that I push this but I'm not waiting for your inputs and finally the the third one is ask ask is what we typically do for every PR all the time right here's a p request I need a permission or a review before it gets merged so this is what we do for everything think about the ability to

do a ship show ask model and again obviously you need to decide what goes into which bucket but those are huge gains if you can um choose the right pull request the right kind of changes where obviously everything got gets checked there's good tests there's good coverage you're not just doing you know riding Cowboy um but still you don't have to go through the longer process all

the time so this is uh a popular model it's not popular in in in action but everyone is you know it's a good way to think about the flexibility problem now let's think about what it takes to build a solution to start powering these things kinds of things remember we talked about knowledge about context about flexibility Automation and about visibility so when when I'm building something like

this I need to think about okay what are the capabilities that a solution needs who runs it where does it run uh how does this scale to work with 50 developers or 5,000 developers there's you know different considerations not just the scaling of the you know whatever engine is running this but also the management questions how do I create a set of rules that kind can apply

to my entire organization and also give some specific groups autonomy to do something else so overrides defaults all of that and some ideas just kind of use cases I can do with such a solution so like a linter for pool request right if the pool request is not enough return it to the issuer with a comment saying this is missing a this is missing B you're missing

a link to your J ticket no need for someone to review this for you this is automatic I can help you find the right J ticket and plug it in but this is basically a cycle where the bot can return the pr to the issuer and say this is missing something we have a little of this with templates and so on but now this can be automated

way notifying the reviewers or the developer when the tests are passing I create a pull request as a reviewer I don't want to review your code before the tests are all passing right sometimes I'm pulling up a p request and CI starts to run it's not even green maybe like I don't want to review your code when it's not ready so notify me at time and priority

PR izing my PRS doing selective builds and CIS all of that smartness um let's look at some additional building blocks around context how do I add context one very easy way is to add labels we use labels and PRS to help the reviewers and the people in the repo find their way so now see you know this this is an example of how you know various uh

um project to use labels with colors to help navigate I can add comments something adds a comment in the pr this is not additional context for the reviewer uh better title description can I help that we are already seeing co-pilot and other players help with creating some context they push some reviews and like an initial review they uh try to help you with improving your description or

creating the pr description so there's assisted reviews in some fashion that's all part of context uh there's badges right people use badges on the repo maybe on PRS to like let you see in a glance the status of things thanks um and obviously jir or whatever you're using for project management the ticket the the why behind list some uh things I can do with with like a

building blocks for helping me with getting the review so what about finding the best reviewer or finding someone to review my code we get some very minimal help from GitHub and similar with suggestions but can we do smarter can you help me understand what I should be looking at or focusing on when I'm looking at the sport request is there a way for me to dynamically change

who should be reviewing like I mentioned maybe pull someone from security or from Architects group or whatever depending on the change that's happening in here maybe a security scanner is running on my code and has some some findings based on these findings pull in the right person from security now they're are our gate now they're a required reviewer that kind of um flexibility around reviews can help

me a lot and also changing the number of required review so going from 1 to zero in some cases that's ship show ask right going from two to one if you're working in a bank or a similar regulated industry there's going to be some we have some customers that require two reviewers on every PR think about all that work can I carve out 10% 15% of those

PRS where we can relax and Only Have One reviewer that's a huge saving so that's just examples of the kind of flexibility you you want in in such a solution um obviously what happens with checks and builds and tests deciding when to run the tests which test to run maybe I can determine from the change that I don't have to run the whole Suite or I can

run a few tests earlier that are more have more probability to fail I want to fail fast smart things about my C and also understanding how long these items take and then what can I do on the pr again we're thinking about automation can I reject it with some comments of the reviewer in like skip a review cycle basically do it for automatically or just approve it

get the right reviewers um and if I'm very brave merge it automatically or close it like yeah you're this PR touches an area that we've decided to deprecate no changes allowed here shut it down or this this part is in a freeze right now for the next two days we're not merging anything you can automate the reject here instead of everyone remember please not merge here this

branch is now protected for two days but these are all very uh relevant use cases and then measurements measure the delay the handoff we talked like you saw some of this with pickup time and others but understanding the Dynamics where are things waiting is crucial for solving the bigger problem of having an efficient and scaled operation around this and eventually happier developers if I get a review

faster and my Cod has merged faster I can move on and I'm typically happier additional considerations when we want to think about building this has to be developer friendly with this is for developers has to be programmable let me code myself out of the problem so extensible prog programmable has to live in my Version Control right the rules how does this uh automation work the definitions they

have to be versioned right it's like almost table Stakes we talked about scaling how do I make it work for a team and a group and a organization or multiple buus in a manageable way and have Audits and metrics around what is it what is this this automation doing for me um has to be extensible I want to be able to trigger something that is unique to

my company slack alert or an API call or something get some additional context from the outside all of these require some extensibility um and there's an opportunity for Community right if I thought about a nice use case or automation or rule for my PRS maybe this is useful for others or maybe I have a problem and others have solved it through some automation I can reuse that

so a community a way to share ideas is also a great benefit when you're thinking about the problem because typically your automation problems around PRS are not unique to you in some flavor someone else has already seen them putting it all together and this is what we have done in our product called gstream uh but again I'm putting all the building blocks here um you're thinking about

an engine to automate the life cycle of a PO request and it has to have some way to program it with a rule file or similar lives in my repo to be versioned it has to run natively in my pipeline so get up actions or similar but I don't want to spin up a whole new method of running automations if I already have that for a lot

of my pipeline has to be reactive to understanding what's changed events in the prmr maybe time based yeah two days have passed now the pr has emerged automatically you had two days to review now merged that's a an an example of a modified show method you have two days to review to comment but after two days unless someone is is objected that the PO request is is

merged so time base changes um and has to have those abilities to decorate the pr to interact with its status to with reviewers with comments mentions all of that people what people do manually should be able to replicate and apply the rules to automate and obviously has to be measurable measure itself measure the pr activity so you know that it's actually helping you and what is doing

for you when we have that we are now extending cicd and we're adding The Continuous merge part right cicd is mostly about builds and machines and we are talking about minutes or hours to minutes that's a huge gain cm is about days remove that weight of of days to get my code uh um reviewed move it from 5 days to 3 Days To two days it's huge

it's about people that's why it's so long that's why it's so there's a lot of substance here and when you can improve the way you continuously merge that's when you can handle that incoming U influx of bought generated code and on top of your you know the developer writing code that is important to get a for this is a snippet an example of uh what a rule

file could look like in this case looking at specific areas in my code if someone touched the Gateway I want to add a specific reviewer by handle that's a very nice simple way in a repo to say this this this person has to review this area of the code and code owners is hard to maintain this is something that's much lighter to to work with and in

general it's all about creating those different review paths or different behaviors for PRS so you automate and you apply that automation sometimes now you have that flexibility to um avoid having Just One path and one route for all the PRS depending on context depending on what's changed you can now make decisions and have different flows so thank you now it's a good time for questions yeah and

seems on SL have one so I will uh read it what are challenges faced in the wop's adoption and implementation because the most you talk like about the benefits or what is good but uh here about the challenges so when you're thinking about like automating your PRS be uh some you know a very obvious challenge that sometimes is raised is hey compliance we are sock 2 type

two or we have some other compliant I need to review everything so guys this is wrong Saku does not mandate um every PR has to be reviewed it does mandate that you have a policy and clear rules and that the rules are protected so it's not everyone can change the rules but it's okay to have automatic reviews or assisted reviews all the way to automatic automatic merge

if this is part of your policy but compliance in many cases is a like a knee-jerk kind of objection um it it can and is overcome pretty easily but that's one thing and then the other is you there are some habits right people are used to working in one way and until the pain is very strong it's sometimes hard to shift on the flip side we see

that developers again this is a an approach that is about letting the developers code their way out of the problem and that typically resonates with developers if developers are owning the process then they are um they're easy to adapt such approaches and apply some Automation and in code based policies if you like okay if the audience has any questions rise the hand okay we have here let's

give a mic hi uh so the question would be this continuous merge assumes that there is a person ready to take the pr right but it would be again a context switch for that person who needs to make a review so how to overcome that J so with continuous merge we're saying sometimes there's no need for a person there's you can automate the the the review process

all the way have it approved or rejected in many cases the pr is not yet there and you can reject it automatically with a nice comment so the issuer can do a cycle you've saved a cycle the idea is that some PRS you can automate and do not require a viewer that's net gain for everyone and then you have time to invest more in the others reviewers

will need to contact switch and there's methods to lower the the cost of contact switching if you're measuring this and you're aware you know one thing one one very obvious thing is to work on making your PRS smaller that will help a faster review but also as developers and as reviewers being aware of the contact switch um leads into very simple Solutions make it a habit to

look at uh PRS waiting for review after you've already done a contact switch which means after lunch after a meeting you first thing in the morning right so you've already paid for a contact switch Pigg it back some reviews on top of that that is a smart reviewer's way of reducing that mental load of switching and we are we all have mandatory contact switches during our day

so leveraging those but that that takes awareness and sometimes the awareness is because you have a number that's showing you hey you're paying a lot for this so awareness is a great way to reduce the the contact switch and then context like I said if I know this P request is going to take me two hours I'm not starting it right now I'm putting time in my

calendar for it and I've saved myself uh the Contex switch of starting and then running into my meeting and then having to continue so there are ways to be smart about content switches and reviews are not going away they're just being smart about them one more question from uh slider can AI help to clear the noise in PRS so the reviewer has a um easier time to

focus on the critical Parts yes so we already seeing AI helping with reviews so it could be about summarizing the code changes let me see what happened here walking me through the code changes like give me a guided tour um sum like understanding the key area like key areas or what has changed can lead into a great input for this automation so if AI has determined that

this change in the code has has an impact on my API endpoints that could be a great rule input to my building block saying okay in this case I need a review from this this some person from this team so AI helps and um it's a great input for some more decisions about automation okay still two minutes we have so I will ask the last question I

see on the slide do you measure automatically merge PRS after 2 days of not being review then what you do if the number is increasing yeah so this question about is about the example that use case saying and again this is these are all possible use cases with such an automation engine about like a TTL for PRS there's two days to review after after that that's your

chance after that it gets merged if you're applying this yes you want to measure it and you want to keep track that no that this is not becoming in like automatic for everyone um it gives you a balance between waiting too long on one hand or getting some inputs people have a chance to look at what they care about if you apply some smart automation you can

make sure that you are getting the right um notifications you can say I care about these areas or these kind of changes at a rule that pulls me in in these cases so I have a chance to review and yes then you would monitor you would use a solution like lineb or similar to monitor that Trend over time and say yes the proportion of those PRS which

got merg because of the deadline is maintainable is small enough it's not a problem that we need to to manage okay I see the last one question and one minute has left so anyway how accurate is the estimate uh Review Time have you ever encountered a quick 5 Min PRS which ended up to required way much more time yeah so it's a machine learning model it's not

designed or uh required to be fully accurate it's designed to give you a a good idea of how long this going to take and yes sometimes a oneliner which looks very simple could be a rabbit hole that you would need to review like very deeply or that you would need to debug in production very deeply we all know that but as a as a rule it's again

it's not saying only spend five minutes here but as a rule it says this is what you what is expected and um there's a good chance you can finish this review quickly so probably you're going to jump on it right now and not wait for tomorrow everyone yeah warm welcome and thanks y for your like topic presentation and thanks for the audience yeah