Christian Boelle - Serverless by Design: The Role of Functions in Europe’s Sovereign Cloud Future

In this talk, Christian Böllert, a product manager at Deutsche Telekom, discusses the deployment and management of serverless functions on their T-Cloud platform. He emphasizes the importance of European cloud solutions and addresses common misconceptions about the limitations of functions as a service outside of major hyperscalers. The speaker elaborates on the architecture of T-Cloud’s function graph, drawing parallels with AWS Lambda, and explains various components such as API gateways, messaging services, and document databases that trigger function executions. Böllert shares insights from their operational experience, focusing on resource optimization techniques, observability, error handling strategies, and the lifecycle of functions. He concludes by highlighting the growing relevance of serverless solutions in the cloud market, underscoring the need for adaptability and continuous learning in deploying cloud applications.

everybody. Can you hear me? All right. Okay, very good. Thanks. Um pleasure to be here um and uh I would like to my the previous session it was all about a push to prod and I would like to talk today about run run on prod. And maybe just to get started, so maybe who of you has have already done some lambda functions on AWS or functions on

Azure and all that? All right. Okay, very interesting and who of you maybe have been already confronted with the question of management of colleagues, do you use already European clouds and is it possible to do so? Ah, okay, that's also interesting. So, I hope I have an interesting topic with brought with me. So, maybe shortly to me who I am. I'm Christian Böllert. I'm working on Deutsche

Telekom or in T-Systems and we are operating the so-called T-Cloud. So, this is the Telekom Cloud. It's a public cloud available in Netherlands, in Germany and Switzerland and um also have last year launched the functions on our side and I this is the topic I brought to you today. So, to start maybe two things I would like to what what I hear all the time is like,

okay, in Europe you can't um get functions as a service when you go away from hyperscalers you will you lose that possibility and the second thing is fast is convenience. And okay, it's nice convenience, but is it really convenient? This is what I will will talk about today. Maybe just to start with. Okay, what do we have? As we have a so-called function graph. This is something

very similar to AWS Lambda um on on T-Cloud. You can you can see here a little bit what do we have? As in the different types of functions, what are the features behind it, what you all can do. Um it it it works exactly the similar way, so you actually focus on the code, you upload your code into the cloud and just let it run. So, you

don't have to manage any infrastructure, any um load balancer, any scaling algorithm behind. So, you have just can focus on it. And today on the on the other side, you see the triggers. The triggers are those which can or which are used to start your functions. Just to give you a little bit orientation because these these abbreviations are not maybe known to everyone. API gateway is clear,

so this is your external interface for functions, synchronous or asynchronous. DMS is our messaging service like Kafka Kafka for streaming into the platform and streaming out. DDS is the document database. OBS is our S3 storage. Um timer and um SMN is actually like notifications. And LTS is the log storage and CTS is the cloud audit trail. So, the CTS is like something when someone logs in, someone

grants permission, someone starts a server. All the activities which are happening on your cloud infrastructure are in the CTS service. And on the [snorts] part below, you see the automations. Also, where functions can be used inside the platform itself to automate behaviors. For example, the the attack events on the firewall to see if things are for example in a in a expected range or are um attacks

except um um attacks happening which are unusual for example. And also the performance and action rules from the APM. For example, APM is a service which monitors down in your cloud architecture. Every component like it could be a Redis cache, it could be a Kafka, it could be a database. So, it's all about um detecting events from there, detecting the um measurements of the TPSs, of the

response times, and so on. And you can um use them to start functions. Um now, what kind of uh do we support today? I think we have most of the common ones um today. The the Docker and Customs Custom I have to say is only Linux because um we we don't support Windows yet, but this is all what you can do. We will also see later on

a couple of examples. Now, beside the function runtimes, there are two I would say more um increments what you need when you build functions. The runtime libraries The runtime libraries for each of those languages bring the basic objects and the basic um um uh but it is the same like also by the other cloud provider. So, you have a context, you have a logger, you have a

request and response object. This kind of things um what you what you need. And the signing libraries, how we call it, these are the ones which you use then for calling other APIs. Also, for example, spawning up infrastructure or reacting to other or calling other APIs on the cloud, this is what the signing utilities will do and help you to uh authenticate against um APIs from Now,

maybe just here a short um a life cycle overview. This is how also functions work in all the other cloud providers typically. So, you have a you have a cold phase and you have a a warm phase. The cold phase is typically where a cloud provider does three things for you. It creates a runtime, it it downloads your code into the specific um execution environment, and then

it starts the runtime. So, this is what we usually know as cold start. Also, from the beginning here until the this the runtime is started, this is actually the time where your code is not executed yet. This is the cloud provider spinning up your your environment. And then, you have kind of three events which you can uh build with your source code, but actually the most important

is the handle. The handle is actually this function which um is doing your main uh uh use case. Whatever you want to do, whatever you have developed in your programming language, the handle is the is the major functions. The initialize is actually the first call um into your code when a function starts. Also, for example, what what most of the people do, they start a connection pool

or they read something from a database into the memory of the function and then and then and then. And this is then available for all do all the handles. And the destroy is then the the the last one when the function is going to to end. Uh that you can even implement something to clean up, for example, your connection pools. This is exactly the same way how

it works. Um And now let's look into the um next part. This is now a example where we now can see how the function looks like. You see the the incoming event. The event could be like, for example, on your S3 storage, a file has been stored. Then exactly the event carries information what bucket is it, what file is it, when it was stored, and then and

then and then. And the context is the context for your function. Also, it means it it holds information about what is your transaction ID, how many tries has been um implemented, or how many tries are currently running for this function. And it has a has a lock handle for you, for example, that you can lock informations and then and then and then. Just a very simple example,

but it shows the the different ways how it's implemented. To be honest, it's also very similar to the other cloud providers. And this is the the the need of the runtime libraries. Without the runtime libraries, you don't have access to the request and response objects. Now, I could go on and on and on and explain how great um the the function graph service on T-Cloud is, but

actually I wanted to bring today a little bit um another element because we have now used the service for a year and we do also use it for big production landscapes and we have learned a lot of things, I would say. And I brought today for you not now what are all the hundreds of features and what are the tips and tricks, what are the problems also

we faced and how we overcome these problems. Also, so the first thing is the resource optimization. So, just to let you know, like in like in other cloud providers with the memory, you actually control the entire resource availability for a function. Also, your network, how many network bandwidth you get, how many CPU power you have in a And then this all regulated with the memory. So, we

have a lot of times the situation that actually too much power is granted to functions. So, that's uh that that that's a simple thing and we found out that this actually really good every month or every 2 months review the real consumption and say, "Okay, there is a lot of potential by reducing the the the memory." Or because every every second and millisecond is counting, correct? Also

for price. The second example I have, >> [snorts] >> the resource the dynamic resource allocation. This is a super cool feature because when you, for example, allocate a function and say, "Okay, this has 4 GB to run." It will always have 4 GB. The problem is that you don't have always the need to process every single call with 4 GB of RAM and the associated network and

CPU resources. So, you can shift actually. So, what we have done is also I have brought two examples here. Um for example, based on the file size or the stream elements you have to process, you can actually regulate the memory up and down and you can do this for every single call. When a function is called it will the function is always same. It's some one time

it runs with 512 MB, that's the example here, or with 2 GB. And that is actually a very useful way um doing the operations to save money. Um also, because the the in in the first example now you see for example this this function runs with just 25% of the it will take a little bit longer, but on the on the on the total you you can

actually save money. Now, with that saved money, there's also one thing what we can do now with it everyone talks about cold start when we talk on functions because they they cold start I would say is the is the is the disadvantage of running a fast solution. On the other side, it's also clear that we cannot avoid the cold start because the runtime is not running all

the time. It's not a container. So now here we have an example. I don't know if it's visible, but you can see this is um a load which we analyzed and we you can see that actually always two parallel functions are running in this in this scenario. There's peak, there's a high and down up and downs, that's clear, but two is always the minimum. Now we we

use here Java especially because Java has the has a heavy lift to start the runtime, to start the entire JVM. Now, you can see that the maximum duration is round about 6 seconds in cold start means 6 seconds is the taking to start up the runtime, and then your code runs 50 50 milliseconds. So this is now this is the challenge. Now we have decided based on

this example that two instances can be kept alive. Means those will never go into the destroyer modus, they will then just be warmed. As we will keep warm. And that then change the the overall maximum duration to 50 milliseconds already. Massive um improvement. It comes also with higher cost, that's clear. As the moment you keep functions warm, um they are always keeping in memory, and this is

cost. But on the you can always take your base load, and the base load can be can can be so-called secured. We call it spare cores or reserved instances, and this is then a useful way to reduce the response times. The second thing I brought the observability. To be honest, there's so many metrics what you what you have to monitor and all that. We just found out,

okay, actually the best thing is to do maybe use a Terraform template for it. And at least from our side, we said three things you always should monitor. The invocation errors, okay, I think I think that's clear, but also the throttling and the concurrency versus max concurrency instances because when your when your functions, for example, is allowed to run in 50 instances, and you then get already

to 48, you you should know that there is something soon to happen that you run into the maximum and the throttling will start. The the next tip we have we have said this was notification policies. This is maybe a little bit a different concept now. This is all about asynchronous processing. We use this a lot when we use Kafka streaming, and they're all happening in background. Notifications

are used to monitor functions itself. So, means a function runs, it it reads a couple of hundred megabyte from a stream, it processes it, and now it fails. Now, what to do? The notification policies are exactly now coming into picture because it says, "If a function fails, start another function." The function would be then the the error handling function, and this function gets two contexts. It gets

a normal context like every function, and it gets the original context. The original context then contains the informations about what was the failing function supposed to process. For example, when a function has to process 10 files from the S3 storage, then the error handling function knows that these were exactly the 10 files which were are to process. This is very useful, but here also I would say

the the big learning from our side is that for logging, the transaction ID is quite important to log both the the current transaction and the transaction ID of the original version. Because if you do log tracing through our through what happens to our call to and the error handling function now comes in, it needs to know I I'm handling now the data from a failed function originally

with ID 1 2 3. So, this was the very big I would say learning which we had that we contain the original as well as the current transaction ID in the logs. Now, versioning and shifting I think I think this is um useful when you are longer in production and you have more and more functions. Because as a the first option which we had was shifting only

specific part of the traffic to the new version. So, [snorts] this is good for for um um grayscale releases or for canary releases. And the the last option I brought here, this is specifically very useful when you have like a beta um um access via the API gateway. Could be a API gateway which is for example really just handling a API call, but could be any UI

call with a HTTP get coming where you then allow in this example here we said if the environment is staging, then the staging could be handled by the new versions and you have then couple of days and weeks in production to monitor the version to the new version to monitor the resource consumption. Is it Is it faster? Um Does it perform better? And then only decide to

go to the next um um upgrade to the Then the next thing what I what I had in the beginning was the hypothesis two, fast is convenient. And to be honest the fact which I was finding interesting was that the fast market itself globally, this is globally now, has increased the last 5 years for 50%. So, this is a lot of growth. Um including all the hyper

scalars. So, this is really definitely growing um more than others. The second thing is that since last year, a lot of analysts are nowadays um scoring fast in the same level than container as a service and databases as a service. So, for example, we work with ISG, Gartner, or Forrester, and they nowadays really say this is equal importance for a cloud provider to have in the portfolio.

And the last thing um which we saw is that, okay, the churn rate in cloud providers is anyhow very, very small. And if if there is a churn, it always takes a lot of years. Also, in enterprises, we talk typically 3 to 5 years in in contract duration. And if there is some movement happen, it never is a is a 100 or or 100% thing. It always

starts small. Right? So, whenever you bring a new a new cloud provider, you start with one application, and over years, maybe things things are changing. >> But, this means when you choose a European cloud provider today, and you don't have a fast option or any function serverless option provided, you have decided this to be a convenience feature for this decade. When we have that um three five

three things now in in place, means that all your teams, all your cloud engineers now have to live and stick with that decision for at least 4 to 5 years. So, I think to be honest, I'm the product manager of the fast solution. Okay, I'm not I'm I'm maybe a little bit biased, but I think it's really not a convenience anymore. Yeah. All right. Now, um also

to to not to do many PowerPoint, I also have a small example for you and also a short video that you see how does it feel to build functions on the on our cloud. Here's a small example. It's a API Gateway HTTP functions um which is coming in. It's called then the backend function and then it stores the data on the OBS storage. So, how we can

do this? I have a short video here. So, here now you can really see how we do it. So, this is currently our cloud environment and you can see uh we are using the UI to also show this because in in Terraform you just have a long script which is running so that you can get to see how does it really feel. So, we're configuring now, for

example, the runtime, the the log collection for every function. Um now this is created. Now we upload the the source code for the first function for the frontend. You can see this here. This is um uh Node.js in this example here and in this example you can see now on this on the right side all the resources, your um entry point for that and all that. The

second thing is now the the trigger so we have to combine this function with the HTTP request to be received from the API Gateway. you can see actually how this works. Now the the public endpoint gets published. And yeah, okay, the time also also synchronized. Exactly. Now the the the third function um or the the step three is the third function which is actually the um function

which now receives the data and will will store it on the OBS. Uh this is again a Node.js function. You with the log collection. Yeah? And also we will see now how the permission management will work and um how to test the function after uploading the source code first. Okay. So, the source code now is uploaded. You can see. Here you can see now the example how

the um data will be stored. It's it's it's a general S3 um call. And now this is a test. So, this is a test function which is just a JSON input for a rating app. We we can see that. So, now you can test on the UI. And now something will happen. You see here there's a problem. The function is not allowed to access the storage. So,

this is why we have to configure now the agency uh agency means that um uh to define the permission. And here you can see now the permissions are to put an object into the storage and testing it again. It works. So, now this is a example now how you can see how we configure for example um the concurrency and the environment variables. In this case, this function

is called with a URN. Also, a function can call >> And for the URN is just a unique ID uh that the function has been uh uh can be allocated. And the concurrency is also updated yeah, to 20 in this example. And yeah, last not There should never be a unthrottled API. We also should be doing this. So, the API we just published, we will activate for

uh throttling policy. This is what you can see here. Um just to give you an example. will be now, I think, 1,000 cores or 10,000 cores per per hour. Will be allowed. Now, you can see this. And this is now the the Node.js application with a front end, with a back end. And if you want to try it out, you can actually do this now. So, with

this QR code, you can try out live now what we actually deployed. So, this means you will see now how the function calls the front end first. You will see uh the page, and you can actually then do the voting. And I hope it really works. You know, the problem with live environments is that okay, but I really I hope it really Good. There was I would

like to mention there was a quite interesting um benchmarking done um in the quarter one this year, um comparing all the major cloud providers in Europe. This is first gateway, us, and the four hyperscalers. And um if you ask me, it's it's it's worth reading it. So, it compares all the features, um also NFRs, um also it doesn't is doing uh execution environment tests. Also, you can

see how much performance you will really get. And also shows that we have some some to do still to do on our side, but it also shows that we have um um in a couple of uh KPIs already beating two or three hyperscalers. I hope it was interesting. if you want, you can also have some questions now. >> [applause]