Boaz Ziniman: Serverless Systems That Won't Burn a Hole in Your Pocket

In this session, Boaz Zinaman discusses cost optimization strategies for serverless architecture, specifically focusing on AWS Lambda functions. He emphasizes the importance of understanding Lambda billing and the factors that affect costs, such as memory allocation, invocation duration, and throughput. Boaz shares practical techniques to improve Lambda cost-effectiveness, including the use of provisioned concurrency, optimizing initialization code, and balancing memory allocation with performance. He also explores architectural considerations such as reducing idle time, event filtering, and leveraging AWS Step Functions and direct integrations to streamline workflows. The talk aims to empower developers to think critically about costs from the early stages of application design in order to maximize efficiency and minimize expenses.

[Music] ladies and Gentlemen please welcome our next speaker boa zinaman presenting the topic serverless systems that won't burn a hole in your pocket okay how you doing it's a bit I I can't actually see you but uh we'll try to have uh some interaction later on so first of all thank you very much for for joining me I'm really happy to be here my name is Boaz

I'm leading the developer relation organization of AWS inia and I was actually born here in Lithuania in in vnus so this is something that uh makes me really excited to speak in vnus and speak about my favorite topic serverless and today we're going to talk about serverless from a slightly different perspective we we are going to talk about cost how can we make our serverless applications or

serverless functions to utilize the resources that are given by AWS in the best way possible and I'm a huge fan of serverless I I've been following this technology since the first day it was announced I run an AWS account that I'm not paying for because this is part of my job and I don't have a single server running in this account I'm very proud about it and

I've been using serverless for everything I can use for that and I want to share with you a Eureka moment I had back in 2014 so back in 2014 I was sitting at the keynote of reinvent Verner vogel's keynote uh and then Verner came on stage and made this announcement ladies and Gentlemen please welcome Chief technology officer of amazon.com verer Vogal good morning Las Vegas morning today

we are going to add these Primitives to AWS with a new compute service I want to introduce to you something it's very exciting it is into AWS Lambda which is an event driven computer service for dynamic applications so think about it an environment with no servers with no infrastructure nothing for you to manage to run the code for you I was a web developer back then I

used to run several teams that were around web development and 50 seconds later I start thinking how can this change completely the way we work so it took some time it took two years for Lambda to become GA and uh then a lot of other parts of the architecture to evolve and now we have a lot of components that are working together in this serverless environment but

this is something that really changed the way I think about software about how we develop and when we build modern applications especially those that are based on serverless there are all kind of goals and aims for that we want to create something that can be highly available manage large amounts of data scalable secure great performance and many many other things and usually cost comes in much dater

stage and there is a very good reason for that by the way it's really really easy to start and it's very very cheap we talk about pricing later but the basic cost of a single Lambda invocation is a fraction of a fraction of a fraction of a scent so usually you don't think about cost uh from the beginning and then you meet the real world you meet

real customers sometimes millions of customers you meet data piles of data and you meet your CFO that suddenly becomes very very interested in your Amazon bills and start asking questions and what I want to do in this talk today is basically try to make you think about cost for serous applications from a very early stage how can we reduce cost and at the same time improve performance

or or at least keep the same level of performance availability in all the great things that we're getting from serverless architecture and what are the tools we have and what are the capabilities of the platform to do that our talk today is going to focus on two parts first part we're going to talk about Lambda functions how to improve your Lambda functions how to make them more

cost effective the second part we'll talk about a much larger architecture how do you do that which components can help you to reduce cost in your modern applications and we're going to start with talking about optimizing your Lambda functions so let's talk about Lambda how can we make Lambda more cost effective what are the ways we have so to do that first of all you need to

really really understand what are you paying for and this is a critical step and there are a lot of tiny details that you should be aware of and I want to walk you through the entire process of Lambda billing how does it work so how many of you are using Lambda I'm trying to see it's really really hard wow that's a lot so I assume you're familiar

with the Lambda execution environment and the life cycle of it for those that are not and maybe I'll be able to tell some new things to some of you when you ask us to run your code basically a trigger something happens somewhere in the system a file being uploaded to S3 an API call or data coming from a firehose or something similar we will look for an

environment to run your code if there is no such environment we will start a process that is called init the init process will take your code download it extract it and that will be the first stage then we will create a new environment to run your code and place the code in this environment and finally we will execute the init code inside your function it's really important

to understand that everything that is outside your Handler the Handler function will be executed during the need stage there are some very nice tricks we can do about it we'll talk about it later then we'll go and execute your Handler and this is basically the first time we're doing for you for this specific environment and if there are additional calls coming in we will keep executing your

Handler over and over and over again inside this environment until until we run a shutdown a shutdown can happen uh in several scenarios one it can happen after no traffic for a specific environment for a certain period of time it's not something we we publish it really depends on the load on the system and some other factors but you can count on it that if you are

calling your Lambda functions over and over again there is very big chance that you will hit on an existing environment the other time when we will initiate a shutdown is when you're updating your code so if you want to force a shutdown on all your active environment just do a code update it could be a tiny one it doesn't really matter we will shut down all environments

related to this code if you've been working with Lambda you probably know the term of cold start cold start is basically this process until your first invocation when we spinning up a new environment it can take sometimes sometime it can be milliseconds it can be few seconds really depends on a lot of factors we're going to talk about and a warm start is when your event is

hitting an existing environment okay and there are differences and it actually can affect cost now for all this goodness the only thing I need to do as the maintainer of this system is to define the memory allocation for my function when defining memory allocation I'm basically saying how much resources I need to execute my code and when I Define the the memory allocation the system will automatically

allocate CPU resources and networking resources based on your memory allocation this is not something I do myself this is will be done automatically by the system the more memory you allocate the more CPU and more network resources you're going to get this is something very very critical when talking about cost because it has a very direct implication and I'm going to show you if you need to

calculate an estimate of how much much CPU resources you are getting uh a rough estimation of 18,800 megabytes will give you an equivalent for one virtual CPU once again it's not really really accurate but if you need to do this Ma on your own you can use this number I don't recommend that there are better tools to try and assess how well your function behaves and if

it has enough resources let's talk about billing now how the building works so I already allocated memory and based on this alloc allocated memory it will take some time for my code to run okay we're talking about the execution of the code and by multiplying those two I will get the duration or the cost of the duration taking this number the duration and multiplying by the number

of invocations the number of events that actually triggered a Lambda will give you the cost of your Lambda and this is something that we calculate all the time of course there are some additional charges there is a charge for uh 1 million executions uh 20 cents for every million executions the additional charges for networking for storage if you're using it you should take this into account but

if we looking only on Lambda cost direct Lambda cost is going duration uh times number of invocation this will give you the cost and as you can see there are three parameters that are tied directly into the cost factor of your Lambda uh function memory latency and throughput and what we will try to do during this session is basically try to reduce all the above and if

we'll be able to to reduce all the above some of them one of them it will have an effect on the cost this is actually not accurate because my goal is not to reduce all the above my goal is to balance all the above to be able to get the best Roi for every function execution let me give you a few examples so let's do a very

basic uh run through basic cost calculation so the cost per gigabyte per second is z is0 do4 $16 uh for every second if we translate it to the cost for a function with 500 112 megabytes allocation it will cost zero eight zeros after the second after the dot uh 83 for every millisecond that those are tiny numbers this is why I said before this is really cheap

and and easy to start with let's say we have a code that runs for 200 milliseconds the basic calculation for that for running 1 million invocation of this code is $166 for your Lambda plus 20 cents for 1 million requests the total is $186 for a million invocation now this is true for x86 want to save some money check if your code can run on arm if

your code can compile and run on arm and today a lot of the code people writing can actually do that and you switch your environment your Lambda environment to use Arm you're going to send to save immediately 20% of the cost just by doing that check if your code is running on arm change your Lambda definition to use Arm save 20% if this is the only thing

you learned from the session you can leave now I Sav you 20% and this is really really basic and this is something you should look into into it but we will talk how to balance between memory trut and try to reduce the cost let me explain you what balancing means so I have a very simple task here for for my code I I want a code that

will compute 1,000 times all prime numbers that are less or equal to 1 million okay not something very complicated slightly compute heavy um not something special so I take this code and I run it in with different memory allocations so you can see that each time I'm adding more memory the time is being reduced and what happened to the price goes up right somebody wants to guess

what happened if I give 124 1024 megabytes can't really see you any guess look at that that's pretty amazing the execution time will be eight times faster than the basic option of 128 the cost will be almost identical this is what I call balance I increase the amount of memory but because I'm getting more CPU in this case networking is less critical the execution time the multiplying

each one of the factors gets me to exact not exactly but very similar price now don't get me wrong if you care only about price and if you have high load on your system you should go with the first option this is the cheapest one and if you don't mind about the fact it takes eight time more to execute the code but if you need to balance

them check these numbers and we'll talk later on how to get to this conclusion with with some tools so after finishing with the basics let's talk about practical things that you can do the first thing I want to do is to optimize Lambda cold start and this is on the latency side now it's important to say if you already forgot we're talking about the first part the

the needit and the first invocation it's really important to say that you usually don't pay for init okay so many things that I'm going to do here will probably improve your code will probably improve your performance but has very little implication on the cost Because unless your un time is more than 10 seconds and few other parameters you're not paying for the init time but still there

are some very interesting things you can do and save some money let's start with something very very basic and I think this is not related to Lambda this is a good a good uh uh software engineering practice reduce the size of your packages okay first of all don't build monoliths with Lambda I don't know if you heard that before but Lambda is a technology that works great

with microservices try to break it over and over again try to make your code as small as possible second make sure that you reduce dependencies and remove those that are not in use if you have a library that you're using in one function and it's not necessarily needed in another one remove it don't just copy paste your your uh blueprint from function to function to function adding

more and more dependencies adding more and more external libraries this will affect the time of the download and extracting your code second you can use a feature called provision concurrency provision concurrency is basically a feature that you're paying for that will tell us to make sure that you have an amount a certain amount of environments that are already warm we will keep them warm for you and

usually say this is a this this cost us money and maybe we should not use it because we're talking about cost optimization not paying more to AWS but if you look carefully at the numbers you will find out that when I'm talking about on demand payment you know the the numbers it's the 20 cent per per per one million requests and this 16666 for every invocation for

provision concurrency you actually have two numbers you need to take into account one is the cost of provision concurrency this is something that you will pay no matter what use it or not and then there is the variable cost of invocation duration which is much lower than in on than on demand if you sum up both numbers you will find out that provision concurrency if utilized correctly

is 60% cheaper than on demand now if you know for sure or very close to that the amount of invocation that your system needs like you have a base that you're working with you can take this base defin it as provision concurrency and enjoy this 60% save only by doing that and once again this requires some work this requires some analysis to understand what is the needed

provision concurrency um of course you can use provision concurrency for uh other use cases not only cost uh if you need to reduce uh call start and many other things but this is something you can do if you know that you have a certain amount of invocations that are needed to your system any Java developers in the room one I'll I'll talk to you specifically uh so

if you doing Java with Lambda there is a nice feature called Lambda uh snapart it has nothing to do with cost actually cost you nothing but it will reduce the time of call start for Java application just enable this and you will uh get the benefits of snap start just another way to reduce your call start as I said no extra cost it won't save you anything

but still worth mentioning the next one is a very nice trick so we spoke about the init code the code that is outside your Handler I'm going to show an example for those of you that are not familiar with that but the init code is a piece of code that runs only once only when we build the environment for the first time now to reduce C start

AWS will allocate a full virtual CPU for your unit code regardless the amount of memory you allocated for a specific function and this is very interesting because if you put more code that you can move to the init you will actually reduce your call start and the price take a look at this code example so this is a very basic code example uh basically this code the

entire code is running from the Handler so when there is a trigger the Handler function is being called it's calling another function basically connecting to an SSM client getting some data and responding with this data pretty straightforward now look at the execution break uh break breakdown in duration 657 millisecond invoke duration 888 total cold invoke time is one and a half seconds this is pretty long for

very very basic code and I'm paying for the bill duration of 888 there are Parts in this code that I don't need to run over and over again all the infrastructure creating the object to connect to SSM client and and so on and get the data is something I can run only once and if I change my code and take them out of the Handler so now

my Handler is only returning the parameter that I got in the init take a look at what happened here so in duration is slightly longer because I'm executing more code but the invoke duration is only 3 milliseconds and because I have more resources at this stage the total cold invoke time drops dramatically this is half of the time when you have the entire code running in your

Handler we're talking about cost build duration is three millisecond okay I can't even do the mat of how much you saved here think about something like runs only a million times a day think about how much you can save with something like that so think about it if you don't need it to run in the Handler if you don't need to repeat this call over and over

again you can put it in the init code do that it will save you a lot of the cost it will make your environment uh more user friendly the call start here is shorter moving on to the next thing and this is actually very interesting this is talking about latency and memory how do I know how much memory should I allocate for a specific code this is

a very important question because as you saw earlier memory affect latency and both of them affect the cost so there are several things you should be aware and you should do so first of all you need to identify what's the optimal memory for a specific function there are several things you can do you can use tools like Lambda power tools I'm going to show it in a

second and and the compute op Optimizer you can use use saving plans if you've been using Reserve instances as saving plans on compute you have the same option with Lambda so you can basically tell AWS that you're going to use a certain amount of compute for Lambda and pay less for that and a very important thing is that you need to check this each time you're doing

a major change so in my previous example when I'm taking a lot of code outside of the Handler I should recheck this again because it might really affect your your cost so let's talk about the tools so Lambda power tuning is an open source tool you can get it from GitHub uh this is a library that's been brought by Alex cazalon a good friend of mine colleague

and it will take your code and execute it with different memory allocation and we give you this nice graph telling you what is the optimal setup for your specific code in this case you can see that I increased the memory to uh 15 uh 1500 and got the best performance and there is no point to add more because adding more memory will increase the price without any

benefit of performance this is something you can run at any given stage and you can use another tool from AWS part of the console which is compute optimize compute optimize you will tell compute optimize to follow a specific function and after a certain amount of time it will tell you what is the right setup for this specific function the main difference between the two is that compute

optimize needs to run inside your account so that means you probably want it to run in production you want real user traffic to this you want real patterns with the open source option you can run it at any stage you can do it during development you can doing without sending any traffic and this is part of what I want you to get out of this talk of

thinking about it in a very early stage so these were few ways to tune your Lambda function to make it more cost-effective now let's talk about optimizing serverless architecture which today is much more interesting because serverless in 2024 is not only Lambda there are a lot of tools there are a lot of options there are a lot of services today that helps you to build a serverless

application and I want you to be aware of some of them we won't be able to cover all of them of course uh time limitations and so on but I'm happy to speak about about them after the talk but I want you to be aware of some of them that will really really help you to cost the first thing you should try to do and that's true

for everything but for cost uh consideration this is really really critical is to reduce the idle time of your application remember we are now talking about the entire application not a specific Lambda function let me give you an example about idle time so this is a very common pattern of fusing Lambda there is a function calling another function calling another function calling an external service and each

time one of these functions is being executed there is wait time so the first function is waiting a lot for the other two to complete the second one is waiting once again the third one is uh the most efficient but still very not efficient by calling an external HTP service for example that might take a lot of time you are paying for this time it can be

milliseconds it can be seconds it can be minutes so you should take this into account and think about it there's another pattern that I ran into just at the beginning of this week when adding some features to a demo I've been running for a very long time I have one function calling a AWS service which is pretty quick it's Dynamo DB it's super super quick fetch something

or write something that takes no time basically and then I do another call to Amazon Bedrock so Amazon Bedrock is a machine learning service that allows me to invoke uh actions against uh llms and in this case I'm taking an image and sending the image as is with a prompt to the uh to bedrock and I want to get a description of the image and then save

it once again to Dynamo DB that that's basically the process what I discovered is that llm calls generative AI calls take some time and I'm actually paying for this time and there must be a better way so we'll have to wait this time because this is how the service is built this is how uh it works with the data I provide to it but maybe there are

better ways to do it and reducing idle time is really really critical for everything related to cost so if you take this code for example uh that checks for a status for for a job and waits until it replies with complete and waits for a second between every call uh that's a waste of money that's a waste of a lot of money think about it running multiple

times over and over again you can take the exact functionality and move it into step functions which is a service of AWS that allow you to orchestrate uh your system and instead of paying for the weight time of the Lambda function just use a stage that is built in into step functions that will do the weight or do the check of the status of this job and

only when it's completed we'll move forward maybe execute another lamba maybe do something else once again this is not a solution that will work for all in some cases this might be more expensive but you should be aware of that you should check it at least to see what is more cost effective for you doing this weight in a Lambda or moving this architecture into step functions

and allowing you to use the features of weight and other things just think about the cost of this if you're waiting now 60 Cycles 60 seconds for a job to be completed that is something that will really make the cost of this specific function super expensive another thing you can do you can offload some of your requests so for example many lambdas are dealing with HTTP requests

sending information into uh into an an API a service over HTP and just waiting waiting for reply uh sometimes there is a timeout you need to re R you need to do a lot of things and you can offload you can offload this request to a service to Amazon eventbridge with specific event Bridge API destinations and this is something that will take the API call and do

it for you now with all the benefits of saving idle time and so on and so forth not waiting for uh the Lambda to complete there are a lot of other benefits that you usually need to implement inside your code like uh authentication support for this API automatic reach rise rate limiting handling and many other things that you need to do when you're dealing with HTTP requests

and instead of implementing this inside your Lambda code which you know already has a price you can offload it into a service like eventbridge API the next thing you can do and this is straightforward is to reduce invocations okay this is basically the trut parameter that we did not discuss yet and I have a quiz actually it's a pretty stupid quiz but still a quiz which is

more expensive executing a Lambda with 256 megabyte allocation memory running for 10 seconds or running no functions at all anybody thinks the one on the left no right so yeah that that's pretty straightforward if I'm not running the code I'm not paying for the code this is the basic promise of servess and the question is how do I do that so one thing you can do you

can do event filtering so I'm sure if you're running Lambda somewhere in your code there is something similar to what's on the screen right now I'm getting into my Handler I'm checking a parameter maybe a parameter I got from the data maybe a parameter that relat relates to date or time or anything else and if it not fits to something I decide earlier I'm just ignoring and

exiting the Lambda without doing nothing okay this is basic filtering of execution in this case I'm basically asking the code if a piece of data I get flee ID is equal to a certain flea and the tire pressure is under 32 if it's uh if it's a under2 I'm doing something if not do nothing exit the why should I get into the code in the first place

event filtering will allow me to define the event with the right filter this will basically tell the system stop processing it don't even invocate the Lambda and instead of this six times I have the right count instead of running it for six times and maybe the three that are this card are super short and very very small billing but still I'm paying for them I will filter

them and run my code only for three times once again this will reduce your throughput and another thing you can do you can do event batching instead of running a Lambda for every event take multiple events and running them with a single Lambda once again it's important to say that you need to check this solution and see if it fits for you because it might work for

your use case it might not but once again I'm trying to give you some ideas and some tools to see how can you do something like that the next one is a relatively new feature and by talking with many developers I know that not many are aware of that using direct integration and this is very interesting to try and integrate into your code so this is a

step function system that is heavily relying still it's using features of Step functions like weight time and many others but I'm going from one function to the other and executing a lot of lambdas here today with step functions we have functions SDK integration that allows us to do step functions directly with services and instead of executing Lambda functions I can go and talk directly to Services I

can go and give a payload to a Dynamo DB table I can take out the response I can take data from one service to another and doing exactly the same thing without uh without using Lambda at all once again there is a cost for every step in Step function you need to check if this is the right thing for you you need to check if you have

enough compute power or features inside this integration to do whatever you've been doing with your Lambda code you might find out that it's not the case but still this is a great option let me take the example I was talking about earlier the the one with uh Amazon Bedrock um that I used the beginning of this week so it's a very basic system I have a an

est3 bucket the bucket is being used to upload images and what I want to get at the end of the day is a description of every new image inside the bucket stored inside Dynamo DB basic B trigger on every new upload to the bucket will trigger a Lambda function the Lambda function will go initiate Amazon bwork uh invoke send the image itself encoded in B 64 together

with the prompt and at the end of the day the description will come back to me as a reply and I'll store it in Amazon dynamodb the problem with this architecture that takes a lot of time takes a lot of time to uh to process the request in in Bedrock it's not a lot it's between 4 to 8 seconds this is what I saw but still this

is a lot of idle time and I can take this code now and basically convert it completely into step functions I do have one Lambda and this Lambda will get the information from the file will process the file but then it will hand off to bedrock bedlock will do its work finish and hand off directly to Dynamo DB and actually this is something that will reduce the

execution time of my Lambda once again need to check if this is cost effective my guess in this case because it takes few seconds to bedlock to finish that it's probably going to be very very cost effective another thing with direct integration uh is something that a lot of us are doing is working with apis and usually the mapping is one to one so we have an

API call uh to a Lambda and Lambda then doing other things you can do direct integration with with a with apis today so instead of calling the API then going to a Lambda function you can go directly to the service once again it's possible that you won't be able to do that with your application with your code because you're doing things inside the Lambda that direct direct

integration what let you do maybe things related to security authentication other things but you should know that this is possible you should test and see if this is something that you would like to integrate inside your code and as I said there are a lot of options I just showing three but take into account that both with step functions direct integration and with API gway direct integration

there are a lot of things you can do to make your application more efficient so I really try to give you few examples really try to bring those ideas that will make your application more cost effective that will allow you to actually reduce cost and use all options all the things that we provide you with AWS serverless architecture few things to to sum up so first of

all I think understanding the payment uh the payment process is critical you as a developer need to know how cost is working with Lambda you need to understand the serverless life cycle you need to understand the factors that affect performance and you need to understand the different pricing models so provision concurrency and saving plans and other things are very very important the other thing is that you

should try to simplify your architecture you should try to think about those things that will make your application simpler less idle time uh and of course less moving components and re-evaluate for every major change this is really critical because it has a direct effect on cost and I think of the most important things is choose your bottles I think this is something I've been using a lot

in in different scenarios uh especially with kids but choose your bottles when you're doing cost optimization if you have a function that runs once a day don't spend your time on trying to optimize that look for those that either process a lot of data or are user facing and uh being invoked many many times those are the ones that will actually move the needle for your bills

so choose your battles wisely verer vas usually at the at the end of his talk is uh saying go build I'm going to say go save so this is my ask from you and if you want to get the slides of this talk they are available in this QR code or go to slides. boss. Cloud this is yet another serverless website I'm running uh that process new

slide text for me and thank you very much for listening I'll be around I think we have some time for for questions if there are any uh thank you very much and I'll be in the ask me anything corer after the talk thank you B thank you so much uh guess is there any questions at the slid platform just guess tell me is there there are no

questions is there any questions in the audience raise your hands maybe maybe yes maybe no maybe yes maybe no guess why there are no questions I have no idea you have no idea they want to go to lunch that's def my first Guin definitely definitely that's a correct answer uh that's a gift for you and I think questions are yeah they start coming actually uh I run

a container in Lambda do you have cost and performance suggestions for this yeah so containers in Lambda uh is is an interesting concept uh for those are you for those that are not aware you can actually take a container and spin it as a Lambda function uh or a runtime environment I think many of the things we've been talking here are relevant to Containers as well so

memory location uh other things um I would always suggest for those running containers in Lambda to check other options as well uh check uh the other run times eks ECS check fargate uh you might be surprised uh by that if you're running containers uh for runtime uh probably uh this is something uh that you can get only from this but check other options this is something I

always recommend if you have a te technology that can run on multiple platforms on AWS check all of them before making a decision which one to do all right thank you uh so last time do we have any questions from audience I see no questions okay do we no oh ah you see what could you recommend to save cost on ald service o nice uh so not

100% related to but uh I do have a talk talking about improving performance uh of non serverless components working with serverless applications and one of my recommendations is use RDS proxy I don't know how many of you are familiar with RDS proxy ALDS proxy is basically sitting between your code in this case might be Lambda or something else and your ALDS server and uh basically being a

proxy uh I know few customers that that saved them one like the cost after implementing RDS proxy was one fourth of the original cost of RDS before doing that uh mainly because it handles connections much better it responds much better so if you're using RDS with Lambda consider that