Thomas Schuetz: Unlocking The Tech Wonderland: Combining GitOps, Platforms, and AI

In this talk, Thomas Schutz explores the intersection of GitOps platforms and AI in the context of cloud-native technologies. As a principal cloud architect and CNCF ambassador, he discusses the complexities of Kubernetes, emphasizing its role as a powerful container orchestration platform. He explains the importance of platform engineering for streamlining deployment processes and how it relates to the vast cloud-native ecosystem that can often overwhelm developers and architects. Schutz introduces the concept of automating problem detection in Kubernetes through an open-source project called Kat GPT, which leverages AI to assist in troubleshooting common issues. Throughout the presentation, he highlights the importance of making platforms user-friendly and emphasizes that successful platform implementation relies on collaboration, community, and understanding user needs.

[Music] ladies and Gentlemen please welcome our next speaker Thomas shoes presenting the topic unlocking the tech Wonderland combining get Ops platforms and AI hello everyone um how are you today perfect okay so um hello everyone to my talk about kops platforms ndi um to the third word in this topic um the main part of AI was the title here right so um I think everyone of us

when when we create some talks and so on we let we let um the generator titles and so was this one um I will get more creative in the future hopefully so um some words about me so my name is Thomas shutz I can pronounce myself um I'm a principal Cloud architect at C so this is a toing company in Austria and I'm a cncf Ambassador so

I'm traveling around the world around Europe and so on I'm talking about Cloud native Technologies platforms and so on um I'm an open source Enthusiast so I like working on open source projects I like using open source project um and I like also to talk about open source projects and last but not least um I think that there should be more technicians and cloud cloud um people

out there therefore I'm also teaching Cloud native Technologies and kubernetes and so on at universities in Austria so um what is this talk about today so um I dealed a bit with some questions in the last few months weeks and so on um and I often hear from students and people out there in the industry cuberes is so complex and I just wanted to discuss this a

bit and talk about it um the second thing I always hear from um many people is that the cloud native ecosystem is so large um and there are so many things out there um what should I use the third thing and this is the main part of this of the story is the mergence of platform engineering and how this relates to the other two questions then um

I want to get a bit more into streamlining and simplifying deployment using GBS and in the end um I want to give you some ideas um how we could find out if everything we are deploying is working so as you might have noticed the agenda is pretty pretty large therefore I'm I will start um so to my first question so everyone tells me cuet is complex right

um who of you thinks CU cuet is complex okay and how many of you think think that it's not complex okay and how many of you are 10 to 15 year uh years um Linux and Unix admins there okay so perfect um so to the first point so just to just for for those who who aren't aware of KU and I think that's the um the other

other fed ones um kubernetes is a container orchestration platform that builds declarative um abstractions for things happening on the system so for instance if we tell kubernetes um create the Pod then it's nothing more than telling a Linux system that it should create a namespace that it should run a process and it should encapsulate something in the namespace nothing more nothing less um the only thing we

have behind it are some controllers and so on which are dealing with the task but in fact um it's open source and everything is transparent right um the second thing and this is what this is about um sometimes things get get a bit scary so we have lots of things to configure we have lots of abstractions in there and with all of these abstractions um it might

get pretty hard to find out what's happening in detail at the moment um but in but also in fact um we can can take a look and in I say let's say in the most cases and then I I barely got got few things where it wasn't reproducible um it works perfectly fine and for those of us who are in the Linux and Unix space for um

a very very long time and this was where I asked before um if you think of ancient times everything is pretty simple so um let's talk about let's think about we wanted to have some high available scalable systems and so on we had or have some such things as custom managers we have S such such things as heartbeat mechanisms and um we have to deal with failover

um and these are only some examples of this and um this also this also scared me in the past so I think it took me hours month years to understand what's happening in my system sometimes um when I was younger and um when it comes to these things also the configuration of all of this was very scary so when you thought when you when you're thinking about

creating a cluster system which is running um which is running on a high number of nodes which should automatically scale which should automatically fail over um and so on um at first it would take you some times to configure it and secondly it would take you lot um it will take you lots of lots of time and braveness um to maintain this so um when we are

thinking about a typical kuas thing and um I know I'm getting into depth there a bit but I just wanted to show you a simple example so in this case I created a simple um Ingress manifest here and I know it's pretty small at it's pretty small here but what you see here is that we have two annotations in there where we say this Ingress object so

a thing which gets into my kubernetes cluster should get a cert certificate which is issued by some issuer and um the DNS entry configured and when we got all of this in cuetes um cuetes or some controllers um which we have in the in our cluster might um say Yes um I am the ing controller and I am responsible for making this service available in My Cloud

store easy as that the second thing we might get is um a tool called third manager or whatever if installed um says yes I'm respond I see The annotation in my in this object um I know that I'm responsible for certificates and I will take care about that this certificate you want here gets available to your application and last but not least um I might have a

third controller in my kubernetes cluster which says yes and uh you need an external um DNS entry so I will go to the cloud provider we configure the DNS entry and everything will be fine in the uh will be fine afterwards um I know this is um this is described very simplistic at the moment so when we take a look at at this picture it might look

like yes qas can do everything and um this comes with some effort to be honest so um for when configuring this it might take about 5 to 10 minutes to get this working think about think about how this was in the traditional infrastructure um space good um but uh when we are dealing with cetes and so on um and this was the same for me when I

when I started with this in the beginning the learning curve is very steep so the first thing the first problem I had when I started dealing with kubernetes was um I wanted to spin up a container and the second thing was I wanted to start to um connect there VI age um today I wouldn't do this um the next thing uh the next problems I had were

such things as um storage um and getting things outside of the cluster so this is what I meant with the learning curve is very steep um but the other and very cool thing is um it's still working on mostly on on Linux machines right so all of the mechanisms we know um below it are still Linux Mech mechanisms so when we want to create a service or

when we want to create um similar objects we always have Linux mechanisms below it which get configured by the controllers and not by us so when we start when we want to create a service it's nothing more than in uh with some networking plugins than creating IP tables rules which make this service available to the outside and inside of the kubernetes clust and the third thing is

you need a really good overview about the ecosystem to get things working so as I said as youve seen in the in the picture before um you might get many products involved you might get many projects involved um and you should be able to find out which which things um fit better to you but this is where I come to in a few minutes so um let

let's go um one step further to the cloud native ecosystem um whom of you knows the cloud landscape whom of you thinks it's too large okay um good so um this is only a part of the cloud native e ecosystem uh so the top part at the moment and yes I agree sometimes it can get a bit hard to to get through it and find out what

what you want to uh what you want to use there um and also when I was in at the University and talked about Cloud native application delivery and so on many of the people many of the students asked me so good there are projects out there like flux there is a project called Aro c um there is a project called captain and so on um many of

them are doing similar or the same things and um why is it this way and what should I do in this case so to the first question why is it this way um it's open source everyone is free to do whatever he or she wants um and therefore many projects out there is the different projects also have different opinions and and approaches so for instance you might

want to have a tool out there which has a fancy UI which is pretty easy to use by the people and has a strong opinion perfectly fine so if you want such tools it's perfectly okay to use them on the other hand there could be more technical things out there where you need in need to know indepth knowledge about what's happening on the inside um you might

want to implement your own opinion in and um therefore you can um therefore there are other projects out there so the different projects in the open source space um are created because of such things because one one project means yes I have this opinion um and then the next project and the next problems will come up where the people say um I canot put my opinion in

the in the same in the same project maybe um so I will create my own one and there from from this such such open source projects evolve um in finally um I I read the book um Cloud strategy from Gregor hope um where he talked about finding the perfect cloud provider for your company and I think that's the same for open source projects and for the for

the um cognitive um ecosystem so you should always try to find these projects which fit the best to your approach and to your company or your system so as as I told you before you can use tools um which are in there which let you configure almost everything perfect but sometimes you don't want to configure everything therefore take the take the easier part so um when when

we come to operating such systems and creating um applications for those systems um I as a pretty old um systems engineer um have always a pretty simple assumption so if I were a developer and nowadays in times of De devops and so on I am a developer um I don't directly care about the ecosystem so I think there should be people in the company which um should

be aware of this which which should uh should be able to help me um the only thing I as an application developer want to do is I want to get my application running and don't want to care about installing and conf configuring its dependencies as far as possible so for instance if my application uses a mongodb and so on it should be pretty fine but um there

should be something where I could say yes for my application I need a m be and where is my connection stream same for any relational database on so um to get there um I want to talk a bit about the emergence of platform engineering at the moment and to start with this I would like to start with a simple definitions of a platform which we have in

the CNC platform white paper I think this was taken from Martin fower um where they say a digital platform is a foundation of self-service apis tools services and knowledge and support which are arranged as a compelling internal product and for me this sentence is very powerful so because for me when I'm using a uh when I I'm creating or using a platform um the first thing I'm

always taking care of is that end users could use this byy themselves without me having to configure everything the only thing um it should be Poss uh which should be needed from me as a as a platform guy or whatever should be the approval of a pool request if they want to um use services from it um the same with tools so um tools should be provided

to use the platform itself um Services can be things as the database as the interface to the database and so on and such things is knowledge and support So how to use the things we have in the platform at the and one of the things which is more and more emerging is that all of this gets arranged as as an internal product so when we are talking

about what what was happening in the pre-ops time we always thought about infrastructure as a project so we said we want to update to Windows Server 2012 um or we want to change uh we want to update from post 5 to six or whatever version is out there um but in the meanwhile we should have some some templates we should have some mechanisms and so on out

there well this is not a project anymore this is commodity this should be done automatically and the only thing um and someone has to provide this and with all of this autonomous delivery teams can make use of the platform and deliver product features um in a very high Pace with reduced coordination so as everything is self-services everything is um um is maintained as a product um it

can be used in a very simple way so um this is also a picture I've taken from the cncf platform um white paper and this is more the conceptual part of the story so um to not go into deep with this we um in the in the bottom of all of this we have some cap uh we might have capability of service providers so for inance instances

could be our Cloud providers hyperscalers as Amazon Google Azure exos scaler how however you want to call them and on top of them we can build platform capabilities so for instance for me the whole deployment part of the story is a platform capability some Secrets management is a platform capability all of the database Services messaging and so on we are needing in our platform our and um

on top of all of this we could have some interfaces where um people or systems can interact with our platform so for instance we could have some apis where a developer could say yes I want to get a database from this and this but this could also be kues custom resources where developers say yes and I need this I need this object I want to create a

p request and simply do this and on top of this and this is the end um we have our product and application teams um which can use all of the things we have in the platform in an abstracted way um to create their applications welcome in um good so what does this mean when if we go to a more in-depth technical level so at the moment we

were at a at a scope where we said yes pretty fine I have Services I have templates and so on but how could I get there from a technical standpoint and um this is only one illustration about things where I could say yes this could be a could be kind of a technical platform for me but this might also also be different in other cases so things

we could we um I need to get into the platform from a developer perspective um I need to get a developer portal um and I'm pretty sure that at this conference there is some talk about backstage or whatever um where you can create or where you can have an entry point into a platform where I can have some documentation where I can have some self-service things in

inside and so on um the other thing we could have are some platform apis such as restful apis or as I said before kubernetes custom definitions and with this we could have many things inside of our platform so for instance if a developer wants to create a new service then he should not uh he or she should not um need to start from scratch right so it

might be cool to have some kind of a platform template where a service template sorry where the developer can say yes I want to create a new goaling service um and I want to get my pipelines configured I want to get the docum documentation configured and so on um and everything runs automatically the second thing we might want to have if we deploy this to a larger

scale um we might also have want to have an environment templates so we might want to say I have not one production cluster but 200 and this should look like the same should look the same in any case and with this we can create some create some kind of infrastructure so um I for myself I prefer to have everything which is configuration and so on in a

kit repository um therefore um for me if it's not in kit it's only a rumor and um I want to have controllers which are taking care about deploying the infrastructure themselves which are environment and which about about configuring everything and with this I get services and um Ingress objects and service interfaces or the services themselves in the kubernetes Clusters I get some things as load balancers um

configured automatically outside of the kubernetes Clusters and maybe if I say I only want to use pass Services I could also um automatically configure all of my pass Services out there and last but not least and this was thing I saw in the last talk um is observability should be a COR part of all of this so when we are are creating such things um we should

be able to find out if some um if everything works as intended how it works and um we should also be able to find um problems if something breaks so um okay and now the picture couldn't get loaded um then I will simply skip it um last but um so one thing I wanted to show here unfortunately it doesn't work so um on the picture there was

more or less a tool store where you could say yes I have some I have some um some sellers which might be the which might be the contact points where I want where I could could get some um proposals on how to build a dog house so this would be more or less my platform interface my knowledge thing and so on um I could have the shelves

by themselves where the wood parts and so on are in there and could say Yes um this is my self-service API in the end and in the end it's myself who fits all of these parts together and tries to tries to build the dog house um um based on my own opin um wishes so when we are building platforms at the first step it might everything sound

like a pretty topic but in fact it isn't so because everyone could say yes I want to build a platform here are my five tools um create a repository create an Aros City application set and just do it but if you want to raise this to a higher level and operate this in a large scale in fact it's all about the people so when you are going

to your platform journey and if you want to get such things successfully working it will not work without people so the first thing I always approach is I try to include and involve the people in my platform initiative as far as possible so when when building such things um I try to approach the people and say yes what do you expect or if you need some additional

service or whatever maybe work on this together because in fact when we are working in one company everything is paid by the same um by by the Same by the same people or by the Same by the same other organizations and therefore everyone should put on the same strings right um the second thing um which might be very important for such approaches is to Inner and open

source your platform Services because you can build platform things um in your in um silently in a room um but nobody will take care about it and nobody will know about it and if you make this publicly a LA and say Yes And this is our platform there is nothing Secret in there and you can use whatever you want there um and if something needs to be

changed or if you want to improve something just raise a pull request um then I think the acceptance of such things will get much better than if you do it in a c in a central um side and last but not least um such efforts cannot work without uh Community approaches so build a community also inside of your company um talk about successes talk about things which

are not working as as good as possible and try to give the people a voice and help them um telling their stories and um one thing was I want to go technically a bit a bit more into depth here is when we're building platforms at the first step and this was also my Approach at the beginning I can build pretty I can build platforms pretty cool I

can go go go there with a blank sheet of paper can draw one square can draw a second square and so on but sometimes the point will come where a deployer first where deployer first system and this can get pretty hard at some point in time so what I learned in the last few years was that deployment influences platforms more than you might think so we have

several action fields in there for instance the first things I stumbled across when I when I built the first platform platform like um thing was I have q Helm charts I have a pipeline and and how do I deal with with secrets and configuration in an automated way now if everything is in kops um there are many solutions out there so there there are external Secrets there

is there are um sealed secrets you have KMS in different and various Cloud providers you should be only aware of them and should build them inside of the of the platform as early as possible let's call it this one um then we have a very a very cumbersome topic and this is a versioning topic so in theory um every service we have in in our platform should

be Deployable independently right whom of you thinks this is this way okay whom of you experiened the counterpart so that it wasn't this way okay the same one so and this was was also what I thought of when I started with my cloud native journey and so on so everyone told me yes your microservices and so on should be Deployable very independently from from each other and

um if you want to deploy a new version just do it and what I found out was um in many cases this won't work so especially if the compan is worked in monolithic monolithic times and so on so versioning and such thing things is a very strange topics topic because you might have various versions which should fit together so you have service versions you have application versions

in the end I had infrastructure versions I have infrastructure module versions and so on um and also if you think about your kops things and so on also the things you you have in kops should not always be um so it might might not always be the wish of everyone to deploy on a system when something is is merged to main so you might also have a

releasing thing versioning thing in um therefore when working in kiub environments and so on tools as um re um renovate and I think the second one was released please got my best friend um to be honest for gitlab I don't have a a a a good counterpart at the moment I started creating an open source project for that but I'm also not sure about this so um

these are these are always things which are hard to deal with so all of the releasing parts of the story another thing I dealed with in the last few years and this gets better and better are the things about infrastructure and application dependencies so when you want to deploy an application you might need some kind of a database behind it um but this database has to be

provisioned some somewhere or and you you need load balances and whatever um and there might be different tooling for this so you might have terra open tof on the one side and you might have a gups controller on the other side so how do you orchestrate them therefore um I will show you some kind of an approach afterwards um but you can also manage many of the

application dependencies in cutis CL with cross brain would also be a valid approach but you should be aware of it um and the last two things we have there are do we want to follow ketes um infrastructure deployment so as I said before with um cross plan or whatever or do we go typical infrastructures code approach as we learned it in the last few years with terap

from open tofu and last but not least what also um has an influence on your on your plat platform strategy is am I using pass services so do I only need interfaces to the to the provider or um am I maintaining the services on my own and this is always a question which you can answer yourself by the capabilities of your team so for instance if you

have hundreds of database Specialists who are who can create um who can maintain stretched clusters across five data five data centers to fulfill the customers um then maintain them by ourselves because at some point it might be cheaper but if you don't have the manow in house um I think this question the the answer is pretty easy to say yes let's use the the um things provided

by provider so um sorry I have a problems with the pictures today um but it works before um so what I wanted to show here is and just imagine it a bit so I've I've showed a very big picture there um where I showed how such things could be deployed so at first I had terraform or whatever which had a which had a pretty a pretty cool

line to the cloud provider API where we say yes and I want to spin up a kubernetes custom and um the second line I had was um I'm also using Terra form or open to um to install the gitops controller on this on this cluster and to install my secrets management Solution on this on this uh cluster because at some point in time I might need secrets

and I will need Secrets earlier than you think because when I firstly access a protected repository um I need my first secret therefore I need to have it in beginning um but what I'm doing typically um is that with this um this is the only thing things I'm deploying bya infrastructures code or mostly the the only things if I'm using everything in my kubernetes clust so the

rest what I will install afterwards is always coming from the from the kops repository and so on this um and this in a way where people could say I only have to create a new path I have to create a config file and the values file for the helm chart and in the end it will work as um as intended um and sorry again for the pictures

so um and everything we are doing here we um we need uh we we might want so we might want to ensure that the quality of of our deployment fits the things um we expect from it so um when we reliably want to deliver software we might trigger our our deployment some somehow so we might have some kind of a peline which says yes deliver it or

we might have an image updator from Argo or flux or whatever which says yes a new version of the image is there please update manifests and then start the deployment process and in the end we will start deploying our services so this can be done by Argo flux or however you call this these things um for the deployment process I always try to follow an approach where

I say yes I want to deploy something I want to find out if this works if it works everything is fine and I can shift the traffic and if it doesn't work then I will try it again so I'm typically following Canary and blue green approaches to decou the deployment part of the story from the releasing and traffic shifting part of the story um to get the

risk a bit out of it because in fact the most riskful um operation we have in this part is the deployment itself and if the deployment itself um went well then the second riskiest thing in there is the application part so and with this I have two or three steps where I could say yes I deployed it everything's fine perfect if not nothing happened hopefully um in

the second case I could say yes I want to to find out if my application works so do some synthetic text tests do some n tests or whatever and in the third step um shift my traffic and say yes this service is now responsible for this deployments and for this we have many tools out there so as I said before for deployment and for deployment itself we

have Aros def flux octopus deploy and whatever um for the whole quality part of the story um and shifting traffic we have Argo roll outs we have flager and also Captain can help us to get this in there and also for validation and this is um you you can always put put in external Services which might help you so for instance when um sometimes I used Captain

to trigger other other things that validates if everything works in my kubernetes cluster and one of these tools was an open source project I'm also working on or uh worked on a long time um and this is Kat GPT and um some of you might wonder what is Kat GPT and why do we need it and um yes this is the second AI part of this talk

um so why did we create Kat GPT so as you might have noticed and heard before um cuetes is pretty complex and troubleshooting kubernetes can be pretty can be really cumbersome so I for myself um I created lots of kubernetes deployments in my in my in my last few years um and I had the problems with missing service accounts not once but but not twice and I

think I think about 10 to 12 so in fact we as s we as as stops people and so on we are able to detect the problems but we are not always on the right path from the beginning and um we are often definitely not the first one hitting hitting this problem so the whole service account things I showed you before so at least um I myself

I had the problem I think about 12 times and um every time I asked other people um many other people also raised their hands if the um if I ask them if they had the problem um and yes we always troubleshooting the same issues right and this is time consuming this is cumbersome and in the end sometimes frustrating so what we wanted to achieve with kid GPT

is why don't we automate the pro uh detection process and also let AI help us solve the problems so one of the most common misunderstandings of K GPT is that it takes ketes manifest um puts everything in a prompt and send it to the send it to the to the a AI provider but in fact what we did with this project was we are collecting problem p

PS in the kubernetes cluster and send the problem problem patterns to the AI provider and ask them for U for a solution for this um so just to give you a short idea on what how this could help you on your on your journey so catg is a can be a service which runs in the kubernetes cluster therefore um you could have an operator which is continuously

looking for misconfigurations or which are not working um and this twice as it seems um and in some cases it if it finds some problems um it will write results to custom resource definitions and can send slack messages if um if it finds something at the moment there are also things going on to say yes I found a problem and I have the secondi in it to

find out well um to to find a solution for it and send it some but this is future thing these are future things um yes the next thing which did not work so um to wrap this talk up because I only have one minute left um what we learned today is still the learning curve is pretty steep but compared to many other mechanisms kubernetes is not so

complex thought so just try to get if you didn't get familiar with it until now um it's a good idea to start with it um and try to find out where the similarities between kubernetes and the typical Linux world world is and you will find out it's not different um the second thing is for developers platforms can make the learning curve less deep so they want to

ship their applications in fact they don't want um they might want to care about kubernetes but in fact um it's not their job at the beginning therefore with platforms it can be easier for them um for me deployment is a fundamental fundamental aspect of the platform so um take this in consideration as rely as possible and last but not least it's all people so take them into

the into consideration when building platforms when creating new mechanisms and so on um yes I also created an article for that this and um yes with this um it's all about the people right so um thank you for your for attention and yes I think we have some questions right yes Thomas we have one question and thank you for the presentation sorry [Applause] I'm and the question

is uh quite simple by the way I would like to say thanks to VL who finally named and we have question from the person not from Anonymous uh which approach is better single architect or group of people sharing with role for the designing the platform or could it work without such Ro um architect can see to us I so um I was a consultant one year ago

so I I have to say it depends so if you are pretty small then it can work with us with a pretty small um if the company is pretty small then everything can work with a pretty small um group of people if the company gets larger larger larger and you need a centralized approach for all of this um you might have a platform Engineering Group which takes

care about the architecture of the of the platform um which deals with some standardization so for instance when we have infrastructure templates when we have server temp service templates where are they and how are they structured and in format um so then you need architects in place but in the end it's for me it's working together so at first yes you can create a u um you

can create the framework for such platforms so to say yes we have this for we have this repositories put your put your um modules there but in the end um you should do this together with the developers and in the end developers should be able to contribute to this did this solve your um answer your question we have one more question any good on Prem or a

based alternatives for K8 GPT I think that it's coming from a people who familiar with gdpr with all other things yes Kat GPT because um in fact normally in the in in the beginning Kat GPT doesn't send any data so if you don't tell it to send data to the to the AI then it won't do anything the second thing we also have is that K GPD

can use local AI as an as a provider so you can run your models locally um and can query can query K GPT on premises and I think there is a company out there from a speaker which comes today also which is also here today um they implemented K GPT into their platform and they are based in Europe and therefore um it's also it's also working perfectly

but at the moment K GPD I think we have nine or 10 Integrations um we are anonymizing data possible um and therefore it's it's an open source project but we are try we are we are trying to take care about the data as possible okay we have one more question uh did you get the real profit with ke GPT compared to traditional monitoring colting systems I think

the profit gets done I think they they fit together pretty pretty good so I know that many many many monitoring companies out there using similar mechanisms in the in the meanwhile kgpt was one of the first projects which dealed with it and at the moment it's for free right so um at the mo you can install K GPT wherever you want you um you can let it

analyze your your cluster and at the at the beginning when we started with the project it the main idea was to make it your companion when troubleshooting things so at the moment when I'm creating deos um the first thing I do is I start the C say k GPT analyze and so on tell me what's what's wrong and afterwards I know if it works or if it

if it doesn't work but in the end I will need a monitoring system which takes care about my objectives which monitors everything alerts me and so on and this is far away from the purpose of kids GPT at the moment okay thank you uh it was really nice and amazing presentation from Thomas and uh please uh Applause to Mr Thomas from Austria thank you and uh we

have a small gift for you