About this talk
In this talk, the speaker discusses how regulatory compliance can enable DevOps practices in financial services, drawing from personal experiences in the industry. They highlight the importance of integrating regulatory requirements into the software development life cycle (SDLC) while maintaining agility. The speaker introduces concepts such as compliance as code and multi-speed organizations, emphasizing that different parts of an organization may need to comply with regulations at varying speeds and degrees of intensity based on their criticality. The session also covers a proposed framework for embedding regulatory controls into DevOps processes to ensure compliance without hindering productivity, illustrating this with examples from the banking sector. The key takeaway is that smart compliance can be a catalyst for enhancing organizational efficiency and quality in software delivery.
Full transcript
[Music] ladies and Gentlemen please welcome our next speaker Spirit on maniotis presenting the topic Regulatory Compliance as a devops enabler hello hello everybody uh welcome to the session compliance as a devops enabler give you a little bit of intro of myself well there's the reason why you see the banks here uh you got to say it my intro uh oh he's missing Technic lovely so I like
to call myself a devops advisor actually it's been many years I work in the DeVos industry I'm currently based in London a Management Consultant close to 13 years of DeVos experience both from Consulting as well as from the industry I have worked with British um Nordic and Friends service institutions amongst others I just refer to Banks here I've done compliance work with all of this going from
the ECB to the EBA and the local fcas in the nordics and Happ that I've published a book a year and a half ago that is called industrializing final Services uh devops m to the content of the session so basically I'll start with a personal story on how I got involved with compliance and eventually that build my dev's career in essence then I'm going to speak a
little bit about the anatomy of the regulat control actually which is the main actor of this session I'll speak about two concepts that are very important when you address regulatory demand into the software development life cycle one is the multip speed and relevance concept of organizations I'm using the example of banking K I'm coming from the finance service industry that's the one that can relate the most
to but I'm sure you can relate to your industries going to speak about the criticality of your portfolio actually how that defines the extent to which you apply compliance and I'm going to bring two small Frameworks one is called compliance as code and the other one how you conceptualize the implementation of a control ending I'm going to give you a little bit of Intel actually on what
to to be mindful on when you design compliance controls so starting with a little bit of a disclaimer we focus on actually we don't speak physical compliance of buildings we don't speak PCI gvr even though I know that sdlc controls can spam into other regulatory uh demand so basically it is a software development life cycle which normally in large organizations is part of a larger it security
framework as it is normally called so bit a story of myself is 2015 November I work in the largest Nordic Bank um based in the nordics actually the bank happens that it's globally systemically important back in the day and I'm head of what we did Call engineering tools brand new team only five months around we read about devops thing is it N9 years ago actually we read
about devops wanted to establish a team in markets technology but I was to to enable devops capabilities it was basically devops tool link and one afternoon my CIO we were demarcated into markets Asset Management retail Banking and so on my CIO from markets tells me to go by his desk and he tell he tells me I'm going to send you an email there is a document there
is an inspection report read it and in the afternoon you're going to be invited in a meeting to represent this um and then I go at my desk actually and I see a report it says European Central Bank in the on page and I'm like a little bit scared what this is about and it was written inspection in it operations so basically it was a 66 Pages
document with what the European Central Bank found when they came to inspect us um and um that I just read it very fast I could relate to it actually so I can tell they've done a really good job what they found it was exactly where the organization is a was and then I go into that meeting and then in that meeting I be told that I'm to
head the stream uh as a program manager and the reason was I was responsible to establish this this devops tooling team as it used to be called almost back in the day and and lead the stream on responding from a Market's perspective into that demand so basically what have they discovered is not exhaustive but it's a little bit all of this we were lacking test automation we
were like segregation of test environments a lot of manual interventions on build the deploy we did not register on the incidents we were not top of slas all the good stuff you would expect a bureaucratic Bank to look like 9 years ago and the most interesting thing is the last sent so basically in the last sentence they were stating that their institution referring to the ECB uh
do not is not confident that we can provide reliable and secure services to our um clients and the society and the most important thing it was the last bit that we we had inadequate capabilities for our size so we were globally systemically important and what I'm saying that is we had to get it done so we had to respond it wasn't a compliance report that you just
leave on the side and and you meet so what we have done a little bit real fast again not exhaustive but automate a lot of things around CD pipelines a lot of things around quality assurance we got on top of a service management started decoupling the portfolio to minimize dependencies and be able to respond to changes and incidents faster faster and everything it took around two to
three years actually to get into the state that they have accepted evidence-based that we had progress and we deliver on commitments and now the interesting thing with this one is that business had his own priorities new functionality Revenue money money money money is a bank actually right revenue is King cost income ratio is what drives everything let's be honest and and most of the ECB doesn't know
what they talk about they were resistant about it actually so we won't deliver new functionality we're going to lose business our competitors are growing and everything and that was the other side of the organization who thought we were blessed in essence and the reality is that we were blessed that they came around and it's primarily for three reasons why that happened having theb around gave us money
okay we changed PowerPoint inversion so that's a little bit sloppy money prioritization to get it done so funding prioritization and then deadlines and we had some really good brains in in the place in that time and basically that has resulted on us getting together and take purely taking advantage of the compliance dedicated money to establish Dev capabilities on an Enterprise level that's a Lial story of my
life that's how I really started doing devops on Enterprise level and then I continue for close to 10 years now now I would like of the regulatory control so basically when you have a regulator inside and they look at your software development life cycle normally they tell you you need to design and Implement certain controls read a little bit this is this is taken it is a
little bit of an average from the ebaa the ECB and the British FCA so basically it says that it is a policy methodology or procedures that show you have adequate means in your organization so you can use your it assess as intended can you raise hand if it doesn't make sense to you so you in it and you get a report and says you need to design
such a thing that doesn't really make sense it doesn't right so basically what I have done through the time is I have manipulated it providing a definition that was accepted by the regulations and that's my own definition actually so it is a mechanism to ensure minimum viable adoption and minimum viable adherence where you either Implement devil capabilities from the ground or where you accelerate DeVos adoption which
is already in the making and to give you a little bit of definition so minimum varable adherence means to adhere to capability let's say stattic code analysis adoption I'm sorry minimum varable adherence means that you adhere to the policy that is built in in the capability and then basically you should be see a dev's control equalizing a devil's capability so basically you flip the definition and the
interpretation upside down and you take advantage of it enableing capability while ensuring it makes you compliant now I brought a little bit the anatomy here to to give you a sense so for me that's taken from my book actually um I'm not here to advertise my book but it happens that the frame are from there so basically I have the full anatomy of how I see dev's
control to be um to be designed and this is a critical path for regression test of an application so that's a capability you need to regression test in never release your critical path as you can see it's it's very rich actually and it covers a lot of areas I won't stand in all of them but I want to speak about four specific ones that I see the
industry usually omitting so one is the value proposition so what do we do it and you remember that we changed the old compliance language to something more modern so basically you need to explain to people why it is happening avoiding the compliance flavor of it so it's not because the European Central Bank is asking us but because we want to deliver high quality services out there second
one is the origin so you need to tell them where it's coming from so they don't think it's coming from their boss or somewhere up in the hierarchy who doesn't know what happens in the organization it needs to be backed up with a little bit of authority that helps in the future to trace as well because if you're a globally systemically important Financial Service institution you're under
too many regulatory requirements from different bodies and you need to be able to trace them on who do you respond with what I'm going to talk a little bit in the last slide about it then is metrics and frequency so basically how often does this control apply to the software development life cycle there things that is oneof the things that are based on continuity actually and they
go together with your release Cycles in every second release and last but not is what is the relevance to your portfolio so is this control applicable to everybody is it mandatory for everybody so what distinguishes people that they need to have it in place and people that they can just omit it going a little bit further now I would like to talk about the concept of multi
speeds and relevance which is very important I guess an example from banking but for obvious reasons so it is important to understand these two concepts when you adopt devops including when you do regulatory work so basically what is relevant for one area that is not relevant to another area for different reasons I'm going to bring an example and how you define different speeds in your organization it's
not the whole organization where with the same speed they're not all to achieve the same targets at the same time if if they ever do it's very important to understand the uh the generation of of these two mechanisms so some are being constructed naturally and natural plar is the business context and the reg regulatory demand so about the the business owners of the application board and who
is regulating actually those applications and the second one is intentionally which is actually Ambitions and ability this is smart people go together in the same team and they just push the boundaries nobody really told them they need to release daily but they can do it and then there's certain characteristics that can Define the demarcate your portfolio in in different speeds and relevance these are some of them
and Regulatory context is the topic of today so if I'm if I may give you a little bit of an example imagine this is a bank and that's an actual example from a bank again sorry for the Mis formating here so a bank is a situation it has sub situations I was placed here in capital markets that's by the examples for markets and no markets better than
reala banking we did have the front office trading really high availability numbers really big demand actually for changes big Revenue um for for the firm really demanding I don't know if you have worked with Traders actually really demanding people actually really aggressive and then you have middle office but you can move a little bit slower it happens mostly after midnight it can be but oriented it doesn't
really go through big changes so you can see here that we have two different speeds a real time that I'm giving an order trade Tesla stocks now then another activity that happens at the end so all the trades come in this a mathematical formulas and calculate the risk exposure of the bank so real time is something I can wait later so two different speeds and two different
scenarios in here and as you can guess when it comes into regulatory work the top priority comes to front office Trading because that's the one that brings most of the revenue that's the one that goes through most of the changes that's the one that is mostly client facing now you got to tell me there's a lot of risk calculations happening here but is where it is in
the this is called the trade life cycle is's go more steps in it the earlier you're in the trade life cycle the more demanding it is for you so basically what do we understand from this example not everybody is going to move with the same speed when it comes into regulatory demand and nobody needs to adhere to the same requirements actually because they're not made to and
this is very important because when you respond to your regulator they ask you for a response and then they they keep you accountable for this response you need to tell them normally Banks do it in waves actually I've done it for three European Banks so far not alone obviously I've been involved you do it in waves and you need to explain them which applications they going to
go in the first waves which in the next ones which they won't be in scope and they need to approve this scope actually so that's very important another mechanism is the criticality of the portfolio normally in a large institution you got to see four different levels of criticality this is what we call Mission critical and systemically critical and especially if you are globally systemically or regionally systemically
or nationally systemically these are super important ones if they are down they cause an issue to the Daily economy so if you have the largest payments provider in the nordex and your servic is down you can push payments through in the nordex it can have massive economic impact actually for the society the second one is business critical bonds they are still important not that important though uh
but they have a distinct uh position in in the business cycle of of of of a bank in my example risk reporting is one of them then you have the ones that are highly available they need to be available enough but not to the extreme right when invoicing is one example so sending invoices to your clients after you receive payments and then you have what you call
Standard as simplified actually can be more complex and these are one the ones that are used internally let's say business intelligence tools by your business users nobody really cares if it is down it's your business people that they won't be able to predict revenue for the next month it doesn't have an impact to the society what you need to look into here when you do the prioritization
is what is the potential consequence from the regulatory impact perspective the one the regulator can tell you we can take your license away they will never do it actually because if you're systemically important they can stop your operation is you're going to end up like with Lion Brothers in the global economy um Their fines definitely and the very important one that is called Capital uh requirements so
if they feel you your business is too risky they going to ask you to improve your core tier one ratio which basically in Bank says how much Capital you have aside to be able to operate for three months with zero income so it can really have an impact on your balance it on top of your uh reputation on the next ones probably you got to get some
fines uh though if you do demonstrate sufficient progress based on the Promises probably got to avoid them though there are many examples of US Banks actually uh throughout the years I think go too many fins and no material impact is for the standard ones right another important Target is the availability so for how long should be up and running the surveys and then how fast it has
to uh be restored when it is down again it goes into waves actually normally the waves are restoration so if we have the data centers of let's say you are in the hybrid Cloud setup nonone bank is running in the cloud apart from the new banks in in institutional ones it's really hybrid so if all the data centers of the country go down which applications it to
come first alive so the economy can keep moving so two examples here two two Frameworks sorry here one is to understand the speeds who is moving fast actually and who is going to move faster in your organization what is relevant to whom I technology dictates a lot actually on how what you can do in devops in the main FR applications you can't be really so flexible as
in the cloud native one and then definitely the criticality from a regulatory perspective as well as from an availability and restoration perspective the economy now getting a little bit I know I know it's theoretical but this is mostly what organizations struggle to understand to be honest and above sorry sorry again for the um for the bad animations um so basically below the red the the yellow line
it was supposed to be highly available those ones will come really late in the process if if ever that was the meaning of it so becoming a little bit more practical this is a framework that I have designed and is not what I'm doing in the afternoon because I don't have time that's I'm really doing with my clients actually it's work I do with my clients and
and this framework sorry for the sloppiness actually um so what happened just to we change laptops and we have in compatible Microsoft um versions of PowerPoint so what you to begin with is you need to look into your entire value stream engineering on how to design and better control so since you have an idea to generate value and organization till the value really ends to to your
to your end client or your end business user you need to look into the controls you want to embed there's a lot of foundation establishment that you have to do I have some examples here we talked about criticality the stuff around threat modeling the stuff about controls inventory so basically make sure that you lay down a very solid foundation that then can build on top it is
super important to have a service model in place so building them it's mostly security tools there but they can be any any technological asset of devops so basically it's very important to embed them into the technology so whoever consumes the technology consume these controls out of the box as well for automation developer experience and many other reasons and then it is it is very important to to
demate them across the continuity cycles of your software development cycle going from development to the service being in production and monitor and logged and tactically embed them across it normally you would like to do a little bit of sift left so you do things as early as possible safe left security safe left quality safe left operations to smoothen the path to the right but then definitely invest
a lot on automating anything that is to the right things like change request Incident Management self-healing Auto healing so basically these are capabilities that end up being controlls ensuring that you can release fast fast recover fast report into sla's impact when you have issues fast and they do count as controls actually from a regulatory perspective and then I propose a set of metrics that they go a
little bit outside the typical Dora metrics you would use the four ones that every company is using or the Dora ones but a little bit manipulated focusing on speed you need to make them relevant to compliance so one thing is to to assess the impact that compliance has to your velocities the when you start adopting controls you got to slow down at the beginning you need to
be able to calculate in monetary terms as well when that investment will start paying off because essentially you got to spend some time to enable things and then you got to move faster in the future um the level of control orchestration so how automated the whole thing is essentially you want to end up in a in a in in almost fully event driven setup across your sdlc
at least for the modern application so basically zero touch to the extent possible the level of Auto remediation so how fast you can remediate things things won't be working so right you don't want people to manually do things right you need to have a way to remediate things very fast so identify X number of vulnerabilities let's say 55% of this x was remediate automatically is very important
developer autonomy no nobody wants to slow developers down and I don't like the word developer but that's the it's it's everybody who is a stakeholder in the devops um value stream but giving people the autonomy um Without Really restricting them when impl controls there's many reasons for that one is that you don't really know their application you don't really know their stack so you need to be
able to give them the authority to manipulate the controls a little bit to fit into what they discussing with the product owners actually did not red tape everybody well you have to do it in certain cases but you need to leave some some autonomy which is actually aligned I call it aligned autonomy in my book so allow them to be autonomous but at the same time make
sure that you're all they are all aligned and then policy engineering so you need to back in the policies in engineered way nobody you don't you don't want big Confluence pages that people go and read policies and what they going to do it should be the events in in the tooling itself actually in your uh service model dictating uh what happens dictating in Brackets so four four
bits to summarize it end to end value stream engineering perspective I don't know if you do value stream engineering organizations so what I'm saying I have this example like you need to be able to follow the controls with your finger and if you stop somewhere something is wrong because you break the flow of compliance service model perspective break the meaning to the technology in an agnostic way
uh if possible make sure you do both a solid foundation as well as enabl capabilities uh which is very important and then be a little bit creative on on how you measure yourself and of course when it comes to measuring yourself um has to do a lot with um um the evidence at the same time now a little bit of a practical example here so this is
let's let's take one example and we say that we have an OP Source scanning policy that we want to make in in uh the CCD pipeline they set an input in output that you need Define in order to ensure what you would call control completeness right so you have everything you need firstly you need to define the logic so basically what is the logic of the policy
that can depend in many different parameters one is the the programming language you're using then what is the acceptable velocity how long should that run to actually I mean if you do have a velocity of 50 minutes for your build time as an example it's not acceptable that this is running more than five let's say because if it takes like 33% of your build cycle then it
doesn't leave enough for the rest very important to Define that thresholds normally you will go for minor medium and major on this one I've seen up to four in that example violation management so if you violated what happens uh do you fail the build um do you need the product owner to sign off a field a build that is passing but it has vulnerabilities and then take
them accountable that Crea an internal let's say audit remark that I know I have vulnerabilities but I to go in production you need to align into all this logic um the triggering event so what is triggering it mostly in the CI is going to build the the build and then in which tooling this is baked in and if you cannot B the policy in the tool something
you've done wrong I haven't come across any policy actually that it cannot be build in in in a tool and I have a I have a proposal later today that if you canot automate a process control sorry then you should probably omit it you don't want to slow down your organization and then all running that example in the build process and then you have some output pass
and fail evidence super important if you can't prove you're doing it you're not doing it and that's not just is saying in in life is is saying from the regulator just prove me that you're doing what you're telling me you're doing um the next orchestrating event so basically this event its success should trigger another event its failure should stop everything in the black and white scenario but
you need to Define once I'm passing this one what is next and then the remediation guidelines in the form of automation so basically I have a client at the moment actually that work together on auto remediation they don't want people to read confidence Pages anymore they want to take advantage of AI Solutions actually and smart things that certain developers have done and created a repository that people
can Auto remediate things as an example now a little bit of guidelines build in in the Tex stack if it's outside this text stack you've done something really wrong um provide compliance as code Frameworks like the one I saw before and you do double click in all the capabilities that they were there um enable it once and consum it many times right and this is a little
bit of a tactic so if some people have done some smart things in the organization just borrow them with pride in essence right and and make them a available to others the reusability is super important avoid the hard coding you need to be able to adjust the controls because your software develop life cycle will adjust your platform will modernize you got to decouple I don't know what
you got to do you would like to decommission it in an easy way I really make it Dynamic avoid the hard coding and I have a great example that I want you to avoid that I see in most of the organization and is really driving bad behavior cruise controls tactical embodiment so basically when you look into your software development life cycle you need to be smart on
what to you embed them um depending on the intelligence you apply on your software development life cycle especially when you have release orchestration but it's different applications running at different speeds and they need to come together at some point it's really important how you do it in intelligent way um allow prototyping in local developer environments so give them early chance to test oh sorry to test what
going to get don't get them surprised when they start using the tool Link in as as a common team right Sun boxing if you probably do a lot of the controls you can enable them not in great magnitude in certain cases low D technological limitations but War them already in advance of what they should expect reference architectures is really important actually especially in public Cloud context at
least in financial services we do have several reference architectures that by default enable controls of course you need to twist and tickle them in in certain cases and then consolidate all the evidence as part of a production redness review so I'm boring the definition from from Google and I like it a lot that's the one I'm using I'm not using the itol one so when you validate
that a service is ready to go to production or is ready to incrementally uh deploy changes in production consolidate dynamically all this evidence and then basically get a tool that has a baked in logic to decide whether you can go to production or not and leave the the only manual step your product owner to sign off because I know that for big Financial Service institutions the regulator
wants to see the name of the one who approved the release it's a requirement unfortunately but again you can do it very fast in that context as well a little bit of guidelines on these too many actually so one is to make them technology agnostic because you got to change Technologies I can tell you now that all the clients in in my firm that we have they
are moving towards gitlab in in GitHub big migrations from Jenkins bamboo a devops right and they okay we did hardcode things how should we do it in a smart way now so don't hardcode them you're going to need them in the future to be flexible collect an initial Baseline right find a way to know who is where um so you can because as I said compliance yes
but we wanton to take advantage of it want to excel the organization so as you collect evidence of adoption and adherence you're also collecting evidence of advancements on devop so you need to go back and and be able to justify the investment because normally that the whole response will be backed up by um a business case uh he he might write it one off in the balance
sheet of of the organization or you might allocate specific money every every quarter or every six months I don't know what the the qbr or annual review process is but you'll have to go back to that Baseline and say okay I gave you $10 you told me I'm going to become compliant and move faster at the same time how can you prove that at least that's how
big finance service institutions work especially in our era uh the money is not really growing in the trees uh this scit if automation is a challenge and had that actually can tell you the story in when I was working at Dan bank I can tell the name right mean only good stories we got the direction from our CTO actually that if you canot automate the control disc
scope it and leave it with me and the co to speak with the Danish FSA and tell them that we understand what you want but you can't slow me down because I'm adding value to the economy so this also so that in my book if you read the the final accountable for all this is the COO in essence so it can go up till there if you
need to have an intelligent and productive Dialogue on what is acceptable and and possible as well uh Dynamic evidence generation definitely you don't want to manipulate them at all actually so really Dynamic you need to be able to populate evidence that tell you what you need without anybody touching them and in some cases a requirement involve your business partners is there applications is there money is there
client is not yours technology is an enabler in essence so don't take them out of the um uh of the equation because I've heard a lot of nasty words from business people that they haven't been involved in such work and saying that you bloody it want to take me out of business in Ence with all these things you're planning who who gave you the green light and
you got to face that especially from areas that are really Revenue aggressive because in certain cases you will have to don't prioritize new features responding to New Markets or in the existing Market to become compliant so they need to be part of the game actually and they need to back you up in certain cases so they can be a good um Ally to have bandwagon your Regulators
that's another important one I mentioned it before normally I mean the the the biggest client that we have now in in at Capco uh my company uh it's I can tell you actually everybody knows it's HSBC HSBC is regulated all over the place actually uh they need to do regulatory bandwagoning so they need to put the main regulator in front and we've done the same at the
Nordic when I was on N actually and then Bagon the others behind the they're less powerful to put it that way and say I'm going to respond to you mate you are the strongest one and then the rest of you see what I'm going to respond to him and just align with each other I know it's not that easy in every context it was rather easy in
the nordics because it's a smaller area geographical area but it's very important because in certain cases you're going to realize that you have conflicting requirements coming your way the one is asking for one thing the other one is asking for another thing and you can't please them both equally um make policies invisible but transparent so make sure we had this saying in the nordex actually make developers
to follow a policy without realizing they are following it and this is where you really succeeded with with compliance work but make them transparent it's different mechanism you can make them transparent and get them to contribute to what a good policy will look like but they need to know how it was design so they need to be part of if you remember the autonomy they need to
be part of Designing the autonomy in essence the people that are to be the end consumers in case you brought de ecosystem vendors everybody actually because what you're going to realize in essence and that is what I've been discussing with a client of mine last week actually it's the second largest British bank and at all them they want to shift their DeVos operating model and I was
saying some things is with your yard some are not in your yard actually I can make you proposal for what is not in your yard so you need to find people that are responsible across the value stream and bring them together because you might want to do certain things but maybe your central hosting team won't allow you to do it and you got to be planning fun
of things that eventually are not easy or acceptable at all and bring your regulat close but not to close actually and you need to keep them updated you need to keep them entertained actually show them you have progress but don't bring them too close I have a lot of tricks in my book again I'm referring to it but I'm not here to sell it as an example
right that don't bring them really close don't ask a lot of questions don't ask stupid questions pretend in some cases you don't understand what they want because if you go for instance and ask for details of what they want they going to make the the requirements extremely tough they got to come with the worst case scenario and in a lot of cases if not in all cases
we know that the Regulators are always a little bit further behind the industry so they even don't understand certain things you do I had actually this is this is an amazing one when I was at dansky bank we had the FSA standing for financial stabilization Authority in Denmark sending their findings they used to call deployments installations that shows you that in certain cases are really behind they
don't even use the same dictionary you use so keep them close show them progress but not too close they don't really need to know everything and then the last one which is the beginning of all the nightmares of compliance actually at least in my experience is do not have C segregation of Duties that's the biggest disaster of compliance actually it's not the early 2000 anymore you can
segregate duties and tasks in your organization if you are that big and is needed in a dynamic intelligent way rather than building walls between your organization because if you build walls between the duties of people essentially you have to build walls in your software development life cycle and that VI violates the whole Automation and value stream engineering uh around the controls ending it if you found it
interesting it's chapter eight is a brief of this one thank you very much uh actually it was a pleasure and just remember that because it's super super important uh if you play smart compliance is a blessing and I really made it i' I've done it three times so it work both three times equal if you played smart that's all had we can kick off the last Q&A
um there are no questions in slido but if you have any please sh uh I can give you a microphone or you can just uh say it out loud one appears yes we have actually uh what was the name of the person Anonymous it is uh yes we have actually in both three cases um that I have I I can be open right is on my LinkedIn
profile I've done it at norda and Don in the nordics and and Lloyd's Bank in in the UK so basically in cases that we didn't see the relevance uh we did push back but we're didn't to push back in the sense we're not doing it you guys are not smart enough uh we did provide them alternative on how things can be done in in in in different
way I can give you an example that was a real Battlefield actually when I was at n they did ask us for pre-production environments across all the critical flows that they were identical to production and simply we told them you will never get identical because we don't have money if we're to build production again it's going to take years and it's going to be too expensive so
basically what we convince them to do is to build islands of pre-production environment so we didn't cover the entire chain never so four or five applications with st maintaining their own ecosystem and only doing with really handcrafted critical flows actually not the entire thing of what they wanted similar topology to production scal down version similar cicd capabilities similar observability capabilities similar autonomous operation capabilities and eventually they
did sign off on that but if you read what they asked for the be at the beginning it was way more different it was way cheaper Slimmer and easier to operate what we delivered and it was a big Battlefield actually it was a battlefield within the bank to begin with um and we were we were really bit revolutionary in markets because markets is markets people think are
the most intelligent one in markets that's true actually my experience working with banks and we did we did have a lot of conflicts within the bank uh to eventually convince the regulator to go a different way now that that's one of the many actually examples more uh okay question is uh how to manage if we have big database yes and we have automate deployment have and how
you in practice just copy this B database to other environment environment Environ staging environment so how manage it I have I have a great story do we have time actually I have a recent story from one we have six minutes lovely so that story let's let's BR let me brag a little bit about my clients actually that's work we've done at Lloyd's Bank last year I'm going
to give you the context and you got to tell me if it's similar to yours so basically what they had to do it was the insurance business so basically they had different policies of their clients and they had to prove to the regulator that they can do simulation of Business Development on those so to prove them that they know what Revenue they're planning to to gain in
the future plus also to do exercises when the rates were changing from the FCA so they can simulate the portfolio on how risky it is when rates are changing the capability wasn't there so what we have done and we thought again we played smart regulatory requirement yes but be smart we also had issues with freshest data in the lower environments we also have issues with production like
environments that the operations could replicate incidents so basically what we have done selective copy of production data it wasn't a full copy because it wasn't needed it was only 10,000 polies so subsetting as it is called staging it in operational data storage that was a production replica again smaller applying different masking rules to the data because it was going to the lower environments based on the different
scenario so there was a scenario of rat testing for The Regulators fresh data for software development life cycle and production replica for it operation transfer them securely through connect direct which was a transmission Channel approved by the bank to transfer data securely and then basically deploy it into different targets un synchronously is there any any close to what you're asking that's what I've done it last year
actually is really recent very much correct so so basically in in most of thisa in most of the case if you don't know the primary keys to put it that way you either build stabs or you complement the fresh production copy with synthetic data at least that's how I've done it most of the times so stoping certain interfaces that give you reference keys and then use synthetic
data to mimic what is the real production reference keys from the different Downstream and Upstream systems as an example production like artificial data which is not in production but but it does help you to connect the lines of sources no oh come on 2025 soon so basically the stops is what you would call we call it Market trade Canon so it's a Canon that bombards with trades
the the system the synthetic data there's many different tools actually uh that you can created it can even be you know how we used to in certain cases create synthetic data and it was really proper data through regression test automation actually so we regression testing the system it was giving results and that was a synthetic data for the lower environment as an example then it has to
be it is very close to the 10 the 10K policies example that I gave actually so so basically the thing is that it depends on the case but what you're telling me now should be happening in the production like environment it shouldn't really be the lower lower ones it can either be in in banking but I come across we call it business development simulation so it's no
really lower one what I want to say with this one on that one probably you can get bigger bus you don't really need to mask it because it's within the segregation of production it makes life way easier with with the example the way you describe it and we have the last question uh theal plan requirement to pre-approve Mission critical service to deplo uh many actually I mean
if you look at the production redness review it can be everything from security engineering quality engineering observability um production likel of the environments uh that they have been testing with fres test data uh Disaster Recovery or if you look at the complete production has anybody of you seen what you believe is a really complete production rightness review checklist all the items in there have a corresponding regulatory
requirement one way or another actually so anything anything to to to answer this question anything that is in the boundaries of security compliance and and sorry security quality and reliability it is mandatory especially when we talk about Mission critical systems and then you can even go to the big extremes now and find the discussions we are having in the UK and I can tell you that openly
nobody has managed to achieve it now that you need you even fail over from from the data center of Google in Dublin to the one of Amazon even more extreme things for systems great so I would like to close the session and uh people could approach you um at the ask me anything corner and then um yes see you at the conference after party which will be
at action uh bip Polo thank you very much thank you every brother
More from this event
See all 58 talks →
Halil Ibrahim Kalkan: Building a Kubernetes Integrated Local Development Environment
45:20
Paco Orozco: Growing at the Edge: Doubling Traffic While Changing the API Gateway
45:03
Viktor Vedmich: Ideal Blueprint Versus Reality for CI/CD Pipelines
46:03
Koray Oksay: Continuous Deployment: The GitOps, The Pipelines, and The Ugly
43:03