Unleashing Event Driven Capabilities With KEDA - Jorge Turrado & Zbynek Roubalik

In this talk, Jorge Turrado and Zbigniew discuss KEDA, which stands for Kubernetes Event Driven Autoscaling, emphasizing its purpose to simplify the auto-scaling process in Kubernetes. They explain the architecture of KEDA, which includes the KEDA operator, metric server adapter, and the role of horizontal pod autoscaler (HPA) in scaling workloads. The speakers cover various features of KEDA, such as scaled objects for horizontal scaling and scaled jobs for job management, alongside advanced scaling capabilities, including dynamic behaviors and multiple triggers. They also introduce new features like the support for additional scalers and improvements in security and authentication processes. Furthermore, best practices for scaling in Kubernetes environments are shared, along with future plans for KEDA, including enhanced HTTP scaling capabilities and support for AI workloads.

Thanks for joining us today. Thanks for being here. It's not always nice to see the room packed. I'm going to introduce myself. I'm Jorge Turrado. There the light the marker. I work as principal site reliability engineer at Smart Git, but I'm also KEDA maintainer and CNCF ambassador. And my fellow maintainer is going to >> Hello everybody. My name is Zbigniew. Very hard to pronounce. I'm from Czech

Republic. I'm also KEDA maintainer. I'm with the project since the beginning. I'm I'm the founder and CTO at company called Kalify. We We do like enterprise version of of KEDA. We can continue, right? >> Perfect. So, what are we going to talk about today? The idea when we were thinking about the session, we thought that probably going to basics, coming back to the beginning was a nice

idea because even there are really nice features. For them you can contact us directly or our fellow maintainers sit on the on the first line. But for the vast majority of the people probably come to basics is nice. So, what is KEDA? KEDA Kuberentes stands for Kubernetes event driven auto scaling. And our goal is super easy. Make the auto scaling in Kubernetes that simple. We are not

aiming to do fancy stuff. Just making the things as simple as possible. And we are a big community and the community grows day by day. So, it's The main contributors are there and a lot of users. It's nice seeing that we have adoption with the publicity effort that we do, to be honest. And the first point is Okay. Scaling on Kubernetes. Does it work? The first option

before bothering you with 20 minutes more of speech of the pitch, I'm going to show you how it works. Just to checking in detail what terrible experience We still see the slides Jorge. So until Jorge fix the fix the stuff maybe there is some question I'm always getting like how to pronounce the name of the project. You can use Keda, you can use Keda, please don't use

Kida. And also please don't use Keda that much because Keda in Czech language I'm from Czech Republic has like a very bad connotation related to poop so If possible please use Keda. Thank you. So in this case I'm going to create the typical publisher. Oh, terrible. Apply, nice it's applied. Now I have the typical publisher and my consumer is a starting to grow. No worries we are

going to go deep we are going to go deeper in this point but before bothering you I want to say that it work. Some rabbit messages have been published and now we are seeing how they are scale. So first the first lesson Keda does work. First not at this slide. I think that we deserve a clap. It's a survival challenge. And it's your turn Zbigniew. Please. >>

Okay. So Keda works and how does it work actually? So I will go through this through this diagram that was recently updated by by Rick who is maintainer with us. So Keda is built on top of Kubernetes. We try to reuse as much possible from Kubernetes as possible. So we depend on on some stuff. We depend on HPA which is the horizontal pod autoscaler. And there are

few components in Keda. So there is a Keda operator. Keda operator manages the resources where we define the scaling. We will go back go back to this part a little bit later. Then there is like a metric server adapter so it is like an adapter that connects to your cluster and you send the metrics to Kubernetes API server through this endpoint. And we have also validation webhook

that checks the checks the resources that we apply. So, imagine that the for the for the example that Jorge did, you define your deployment and then you define a resource where you define the scaling. KEDA KEDA operator will realize that there is a new resource, so it will it will validate it and then it will try to open the connection in this case to RabbitMQ to check

the queue and based on that provide metrics to to to the to the Kubernetes. So, for scaling from zero to one, this is done by the operator. From scaling from one to N replicas, this is done by HPA, the horizontal pod autoscaler, and we provide custom metrics. Uh thank you. Uh and there are some other other things, but the main main thing is that we define the

scaling metadata in the scaled object or scaled job and we try to do as much as possible in compliance with with Kubernetes. Okay. So, scaled object. So, scaled object KEDA has basically two modes, scaling horizontally scaling workloads and scheduling jobs. So, this is the first example. If you want to scale your application horizontally, adding more replicas or using the number of replicas, you use scaled object. There

are let's say three main parts. First part is like the obviously the the target, so you would you define what deployment I would like to scale. Then there is a section where you define configuration, minimum replicas, maximum some advanced configuration. And the last last part part is the trigger section. There you define all those different signals, different triggers that we use for scaling. Because what is the

difference between like the HPA and KEDA? With HPA, you use you can use CPU or memory metrics for scaling, but if you want to use some other signals like the queue length or I don't know some some custom metrics coming from some you need to plug it somehow. So, this is what KEDA does and you can combine multiple different signals and do the scaling based on that.

The second important object or resource is called scale job and scale job works similar way. It also connects to the signal resources, get all the signals and metrics, but instead of creating new replicas of the deployment, it schedules a Kubernetes jobs. why is it good? Because if you think about some, you know, long-running processes running some computation, if you use horizontal scaling for that purposes, it could

take hours, you know, for the process to finish. So, imagine we consume some stuff based on some metric, the replicas are are being, you know, scheduled to some, for example, 10. And after some time, after an hour, HPA will decide, let's scale down the the number of replicas because the metric is no longer there. So, for this kind of purposes, the scale job is a perfect perfect

way because you create a job which has a some start and an end, right? So, you can think of it as a cron job on steroids. So, instead of, you know, creating a job statically or in some cron schedule, you create Kubernetes jobs only when you need them based on some specific metrics. Uh the last last thing is is related to security, authentication, and this kind of

things because this is also very important, right, for us? Because we need to connect to these different sources to RabbitMQ, Kafka, whatever. So, we provide lot of different options how you can specify the authentication, environment variables, which is something, you know, you shouldn't use, secrets, identity providers, vaults, you name it. So, there are rich set of of features that we support on this front. Okay, so this

kind of works, right? So, >> It works. >> So, let's get, you know, to some new features that that are recently introduced. Jorge, would you like to take over it? >> For sure. Just because before jumping to new features, just quick addition to my colleague's explanation. Please, don't try to use secret directly placing them of a scale object or on a scale jobs. We are quite how

to say um strict to use a polite way in terms of the the content of the uh Mani Meta data. If you want to use a secret, you must use trigger authentication or cluster trigger authentication. The password in the previous slide was just for demonstration purposes, but it's not accepted there because we try to build things secure and safe. So, it's relevant to uh at that specific

part. In >> Yeah, baby, one more thing, sorry. Uh you also define the the trigger authentication in a let's say single place and in the then it could be used by multiple multiple uh you know, workloads and deployments in the next space or in the cluster level. >> That's correct. There have been >> Go ahead. Go ahead. >> Thanks. One uh well, one feature that happens every

year and this is not nothing new. Uh it's a new scalers are added. Currently, the portfolio of the scalers is more than 70 70 different sources just uh one click away for or one manifest away from your workload. In this case, they have been for Kaho, they uh another for Kubernetes resources, Sumo. I don't remember the whole list. Another scaler for Solace, for other topologies, Splunk. A

bunch of them, you can check it. But, there are new features super nice. Another new super nice feature is the the fallback for value metric type because do you know which are the metric the possible metric types of Kubernetes? How many of them? Stand up if you think that there are two. Only two? Not all only three? Not all at once, please. No, don't don't be shy.

Well, no worries. I'm here to teach because I teach and enjoy. The point is that uh there are different metric types, average value average value, value, utilization. There are several of them and historically KEDA has supported fallback for average value. Fallback means that when your upstream is down because any reason, doesn't matter, usually Kubernetes stop auto scaling. In case of configuring the fallback, you can say, "Okay,

I don't have upstream information, but I want to defend my workload scaling to X instances." It's something that you >> Upstream in this example is for example the RabbitMQ that you use. So, the RabbitMQ is down and you won't like to solve it, right? >> That's correct. Thanks for the clarification. In this case, you can also use this feature for value. Also, we have added dynamically behaviors

because maybe you wouldn't don't know how many replicas you should have, but you say, "At least those that I have. Increase if for any reason is needed, but never ever decrease or keep it as they are or only decrease, but never increase." Those kind of dynamic behaviors are another super nice feature that we have added. And probably the less important, but most interesting for me is the

support for a new architecture. If you are running on ABM machines, you can use also KEDA because we officially support that the S390X Fascinating. >> Cool. Okay. >> it. Do you want to introduce best practices or I can? >> Introduce them. The scaling. First of all, let me the marker, please. >> You know, there is just one screen, so you shouldn't use it because >> Ah. >>

You need to scale yourself to >> Maybe I can Maybe I can't hear. No, Oh. Oh. No. No. No. No, I will I will try to do my best. First of all, there are two important things two important scaling phases in Keda. One is the activation phase and and the other one is the scaling phase. Activation phase means when your workload is at zero or must go

to zero. So, to start from to zero. And that is managed directly at operator, but as we fully rely on HPA, we need to also give customization capabilities on a scaling phase. Those things are really best practices because if you are working with Kubernetes, you probably have seen that scaling out is super quick, but the scaling in scaling in is not. And those things that you have

to take into account is scaling out adding connections to a database can solve your API problem killing your database. Because the amount of connection for instance of a Postgres server are limited. So, maybe you should think, "Okay, instead of adding instances every cycle every 15 seconds, let's set cool down period of 1 minute before adding instances." Those kind of things are in especially interesting on those scenarios

where just adding instances without any sense, "Yeah, just give me more." Like with a bucket, with a bell, no problem. Are situations that doesn't fit for us. In those scenarios, you should configure the HPA behavior. >> Yeah, yeah, I can maybe add maybe few qualification points there. You are completely correct. It's really like a good practice to spend some time configuring this scaling behavior if you are

scaling some sensitive workloads as Jorge mentioned the database related because if the metric is going up and down up and down, you want to smooth a little bit the transition and you want to be sure that you are adding the right amount of of replica. So, please spend a spend some time configuring that. Um the What is the next stuff? Yeah, cron scaler. This is very >>

Ooh, are you not prepared for the session, eh? >> Oh, no, sorry for that. You see the the intro slide. So, um cron scaler. Cron scaler is very popular scaler. It allows you to scale based on cron schedule, obviously. Uh and uh the typical question we got, I want to scale to zero, you know, during some time. But cron scaler, you know, doesn't allow me to set

the minimum replicas to zero because for some reason, we don't know why. It It is because of how the cron schedule actually works. If you think about like how cron scheduling work, it's very hard to any in combination with HPA and all these kind of things, it's very hard to do it. So, instead of specifying the zero state, you should do the opposite. So, in the minimum

replicas on the on the scale object level, you should specify zero, and in the cron schedule, specify when you don't want to have the zero. I don't want to spend much details on the technicalities, but please do do the approach. It's very straightforward. The other thing is related to the stuff that Jorge mentioned with the scaling behavior. As he mentioned, there is like the activation phase and

scaling phase. There is a setting on a scaled object which is called polling interval, and I've seen it you know, many times that people are tweaking the polling interval, and they are thinking that they are, you know, controlling how often KEDA asks for the metric. Uh this is correct in a way that but it is related only for zero to one scaling. Once there is a one

to end scaling, it's been taken over by HPA. And HPA has its own setting. It's on the cluster level, uh usually defaulting to 15 seconds. So, for the polling interval, KEDA asks for for the new metric, and at the same time, HPA also every 15 seconds asks for the metric. So, So, if you don't if you don't scale to zero, you don't have to deal with polling

interval at all, so you don't please don't specify it because it will it will save you some some some traffic from KEDA. Especially in the larger deployments, you can have a lot of lot of for metrics for several services. Uh the other other cool setting maybe a little bit hidden on a on a trigger level you can specify that you would like to use caching for for

the metric values and this is completely related to the previous stuff. This way we can we can basically omit the calls from the HPA level. So those is each 15 seconds will not hit the the metric in the for example in the RabbitMQ but it will just use the value that's cached on the on the KEDA level. So then KEDA really like ask for the metric only

during the polling interval. So this is way how you can tweak the settings a little bit if you are overloading your services with too many requests from from KEDA. >> Just to give you before that just to give you an example is quite common that external services had rate limiter. In that scenarios is a good practice just setting okay instead of pulling a lot of times per

minute if my metric doesn't needs to be checked less than once per minute set pulling interval one with 60 second and cache the metric because in that a scenario you can ensure that only one request is done. In the other case you will have four requests for a per HPA controller checks plus your pulling interval result of request which are request of result. Okay, another cool cool

feature maybe a little bit advanced. As I mentioned you can specify multiple triggers in a scaled object like a target multiple for example RabbitMQ queues or you know different services and the default setting on on HPA again we are relying on HPA is that if there are multiple metrics for the same the largest value wins the basically calculation. So for example if trigger tells us to scale

to five replicas trigger two tells us to scale to two replicas the first one will win because it's larger. To overcome this problem because sometimes you would like to use average or maybe the minimum or maybe some other other computation we come up with a with a scaling modifiers. So before we send the metrics to HPA, we can do some cool stuff about it. So we can

do average, we can do even some complex stuff like the nested conditions operators. So the So the possibilities are basically limitless. Uh one of the examples uh uh or one of the questions we got from from our users. It was like, I have a service and I would like to, you know, be always over provisioned by three replicas for the for the specific workload. Uh how can

we do that? So what we did, basically we write a query in the scaling modifier that it will all the all the time add three more replicas to the to the specific state. And I believe we have a demo about this, >> Yeah. And another quite a interesting use case of this is the typical one when you where you have the typical consumer a worker or something

so doing something with the database and the database is responding slowly. So the queue is growing but not because of the queue, because of the uh target the store of the metrics. In that the scenario, if you scale the workers the database will be worse, even worse because it's overloaded. With these custom formulas, with these the scaling scaling modifiers, you can use my database as source A,

my queue as source B, and if the database consume is growing, I decrease the value in a in an inverted proportional way closing to zero to a scale in the worker if the database is overload. Those are the the the limit is your imagination in this regard because this can be extended for any use case that you have. For this scenario, and now I'm going to show

you the the scale object we have a super super Sorry, yeah, the color theme. I remember it. >> No, who's not prepared, right? It's not me but you. >> The life, it is what it is. Not surprise. >> You saw what you wanted. >> My eyes are bleeding. Well, in that scenario, you can explain it with you you have the marker. >> But again, there are you

know free spins. >> In this case, we have the thanks. That's why you are the big boss. >> Yes. >> If you see, we have two different triggers and each trigger has a name. The name is an arbitrary value that you give in order to identify the variable in the formula. In this case, one variable that is named as cron and the other that is named as

reference and I'm just checking cron for references. In this is case, as cron is two, this reference for two. So, I'm modifying them. I'm mixing different values, different sources and I have used only two triggers, but you can use all the triggers that you need. You can use all the trigger portfolio that KEDA supports as a metric source to build your own auto scaling solution without having

to code any single line of code, which is a really nice option. Considering this, you can imagine that as soon as I scaling modifiers, as I scale to one my reference workload, as I have said reference for two, I will have two different workload. My reference is a scaling automatically. The other workload has a started because it's the activation phase and in 30 seconds or 20 seconds

more or less or or even faster, the HPA has taken over about the auto scaling and now I have my second instance because the formula says okay, reference for two. >> Awesome. Awesome. That's great. Great job. >> It worked. >> So nice. And I removed the slide before starting the demo this time. >> Okay. Okay. >> Okay, this one is controversial. I'll take it. So, the very

typical use case is that you are starting with auto scaling, you are using Prometheus to for your observability tool or DataDog or Dynatrace or any social like that. So, we have the metrics, we have the custom metrics. So, why don't we use, you know, this as a source for for the scaling as well? It works. It works until certain certain point. When you hit certain scale, so

the number of resources you are scaling, the number of scaled objects, then you might start overloading the Prometheus instance or a DataDog rate limiting and you can pay a lot of money. So, you should think about about the the problems that can arise. Again, for small deployments or maybe for deployments that are not so sensitive to to time or to like a speed of of scaling, you

can use it. But, I've seen a lot of users for video large deployments or with with with metrics that they will try to do the scaling as soon as possible. So, they reduce all the intervals, all the polling intervals, everything they used to 1 second. So, they were polling the Prometheus every second with thousands of, you know, requests because they have thousand workloads and the Prometheus couldn't

handle it. So, because Prometheus is great for, you know, storing the storing the metrics and, you know, evaluation the metrics over some time. But, for auto scaling, you don't care about the metrics that happened, you know, months ago. You are you are interested in the scalings that, you know, happen maybe in the in the last minute, right? So, the So, there are many problems. One of the

problem is the If you look at the diagram, uh the blue part is the our workload, our application. We are exposing some custom metrics. So, we scrape the metrics to Prometheus, DataDog, Dynatrace, whatever. So, this is like a one one, you know, period. It could be 30 seconds. Then again, KEDA needs to scrape the metrics from from Prometheus to know like what should we do. And as

we learned that there is a polling interval, there is like the HPA mechanism, so it could be up to 1 minute. So there the the delay could be 1 minute there. Again, for a lot of workloads it's fine. If you have some time-critical workloads, this might be might be problematic a little bit. The other problem is that you if you especially if you are hosting the Prometheus

outside of the cluster, you send a lot of traffic out of the cluster and then back. So again, it's like a kind of silly practice in total. If you have the metrics in the cluster already, why don't you why do you send them over there? Extra spend, extra storage, etc. etc. There is a blog post written about this problem. Again, I'm not saying that Prometheus is bad

or this bad bad approach. If works for you, that's great. But if you hit certain limit, you need to think about the the consequences. So you can try to tweak the the cache metrics caching. You can try to tweak the the polling intervals or you you can try to think about how can we replace like this specific metric with some metric that's present natively in in Kubernetes

or >> Or or even deploying another specialized Prometheus Prometheus only for this purpose with shorter storing for with a say longer scrap a scrape period, that hit that fits your necessities. We are not saying this doesn't work. We are saying if you plan to have this at a scale with 10,000 scale objects, >> You need to plan properly. >> you need to think about your setup. That's

all. >> Exactly. All right. What's next? >> The next is the future. >> It's great. >> Future is now. We are There are some topics. All these next topics are more ideas that we are >> Jorge, stop. I don't see AI there. >> No problem. No problem. If I have solved title AI No, in Spanish >> AI is everywhere, right? So >> Fancy AI. >> Awesome. So

now we have AI with Keda. >> Nice. >> Uh this one. >> Yeah, we have five last five five minutes, so speed up. >> So, yeah. We are planning to support uh scale just suspending a scale jobs because it's a feature that Kubernetes does and it's a really nice feature that we want to explore. There is an issue there that you can follow and get more info.

Also, there are some that discussions on Kubernetes uh on different Kubernetes places about uh restarting and checkpointing workloads directly and container runtime layer. It's something that doesn't rely or doesn't depend on us, but we want to be aware and evaluate those kind of >> Yeah, so you can think about it like uh instead of scaling to zero, you can just the idea is like to just to

freeze the freeze the workload and then instantly start scaling, but this is not something >> that we can do. It's something that we can implement or integrate when it's ready at a runtime level. Also, better support for GPU. It's something that we are working on because GPU for well, AI is there. Underlying of that line is AI. The point is that GPU is another scaling metric that

we need to consider and we are checking which is the best way how is the best way to do A potential V3. I know that you hate that idea. I have to get an agreement from other maintainers, but potential V3 is a something that we are working or thinking about and also about promoting our CRDs. I think that three years as graduated project eight years of a

project it's a good moment to think about promoting to beta. >> We have to be conservative. >> one year. >> And whatever you think what whatever you want to say about AI fancy topics. >> That's all AI fancy. That's it. >> Perfect. And the last but not least if you want to scale based on HTTP workload, just to give you the hint because it's another topic for

another whole session. >> Actually, there is a session tomorrow about this Trivago and our firm maintainer, so go to the session. It's >> one of the latest sessions in the in the in the day, but it will be great. >> Yeah. Because I think that we are almost in time, so just to give you a quick introduction. We should we are working on supporting HTTP workload scaling

also from one to zero. The project I know that this is the eternal promise, but this year is the the good year, is the chosen year. >> After we free and we won on >> After yeah, after Red Hat has decided to invest to commit capacity. So this is the year. Just to give you an example, this is not the right example. Go to the repository. This

is more or less the archi- >> Skip this. Yeah. >> Yeah, perfect. Let's so let's go to the demo. That's correct. Let me try to open the demo. >> You can think of this as AWS Lambda but on Kubernetes, so >> it's loading because there isn't any workload and bam! Suddenly, it has a started and it's something that I can show you if I go to HTTP

demo. My workload is there. 15 seconds. It will it scale it was or it will be it scale to zero as soon as the traffic ends, but if I check it before the workload is deleted as you can see, it's automatically refreshed because the workload is already running. So which is the magic here? The magic is that HTTP adapter of Keda is whole is intercepting the request,

is holding the request till the target backend is up and running and then the request is passed to the target backend. Why? Because otherwise the experience is 500 or 502 try again later. That's not a really good user experience if you fail terribly on each code request. So, more or less that's the goal of the app. >> Awesome. So, thank you very much for the for the

session. If you have any questions, feel free to ask us. I have two t-shirts for good questions, so please challenge us. Yeah, show this slide probably. So, questions? Anyone? There's a question maybe Yeah, thank Can you go to the microphone? There's a microphone. >> That works, too. >> Thank you. Thanks for the talk, by the way. So, I was wondering with the HTTP-based scaling, how does that

differ from Knative? >> It's a similar capability, but with Knative you need to deploy Knative This is This works with the traditional deployment. You can also combine it with other KEDA scalers. So, it's like a providing a similar capability like Knative, but in a KEDA way, so you are using only one auto scaling solution for for your setup. But, it's similar similar similar, let's say, similar. >>

It seems like it's almost solving the same problem. >> Yes, exactly. Exactly. >> Okay, thank you. >> I would say that our goal has been more focused on async on event-driven. Knative has been more involved in direct traffic. And when we started to manage traffic and they have a started to try event-driven setup. There are things that overlap, but I would say that they have been different

projects for different purposes. >> Any other questions? Oh, we are out of time, so we can take this offline probably. Thank you again. >> The only question, the only the last >> Yeah, sorry. I don't know how to get to You talked about checking, for example, database and depends on database load. You can scale down if even if a queue is growing and database is overload, you

can scale down. But I have question, do do you have the same possibility to to implement for some API because for example, some API service can be overloaded and it can be the same problem with database. How you we can mitigate this? >> I will give you a very quick answer. If there is a metric you can expose about anything, it could be used for auto scaling.

So this is like the general answer for basically any question. If you can expose the metric about the problem, we can target it and use it for the scaling. But continue. >> Yeah, the idea >> So it's must be just push it to Prometheus or somewhere and Keda can read from Prometheus and use the same >> That's correct. Yeah. >> Target a current CPU usage for the

database or for my API or for whatever. The idea is that with custom with scaling modifiers, you can create your formula to decrease the the result based on your database or your CPU or the metric that you want. You can build your own formula instead of just adding, you can increase, decrease, customize, whatever you want.