OpenCost - Cost and Resource Management Deep Dive - Rajith Attapattu, Randoli

This talk introduces Open Cost, an open-source cost monitoring tool for Kubernetes, presented by Rajit Tatawatu, founder and CTO of Renoli. The speaker discusses the architecture of Open Cost, which aggregates cloud pricing and usage metrics to provide detailed cost visibility across multiple clusters and clouds, including AWS, Azure, and GCP. Open Cost allows users to manage their Kubernetes expenditures effectively by breaking down costs at granular levels, such as by pod or container. The session also outlines new developments in Open Cost, such as improvements in testing and the upcoming promless mode, which targets performance issues. Additionally, the speaker invites participants to contribute to the Open Cost community and highlights the opportunity for maintainers to engage in the project.

Awesome. Okay. Um, happy to see all of you. Like I said, I wasn't uh, you know, wasn't sure how many people are cared about costing Kubernetes. Seems like a lot. Um, I don't know if everybody's penny pinching or people have become wiser, but either way, I think it's a good thing to pay attention to. Um, um, thanks a lot, uh, for giving your time today. Uh uh

CubeCon's a very packed schedule. So I'm glad um all of you made the time here. Um and it's a pleasure and a privilege to come here and talk about uh Opencast. You guys can hear me at the back. All right. Awesome. Okay. Good. Um how many of you have heard about OpenCast? Yeah. Okay. Good. So we're probably doing a decent job about this. Yeah. Okay. Good. All

right. So um let's get started. My name is Rajit Tatawatu. Um I'm the founder and CTO of Renoli. uh render is an observability platform um that also has uh cost visibility because we believe it's an essential part of uh and uh I'm also a maintainer of the open cost project and I'm here to talk to you guys about um you know opensource cost monitoring for Kubernetes. Um

how many of you showed up at the open cost booth today? O we need more. Okay, this time we got three days. Um, so just drop by and uh we can only offer you a sticker and um an a warm chat, but we'll take it. Yeah. Okay. Awesome. All right. So um what I want to do today is to introduce um open cost uh quickly talk a

little bit about the architecture and also talk about what's new in open cost and then um talk a little bit of the road map and hopefully entice some of you to um contribute to open cost. Uh that's how my journey was as well. So you know when we were building um we needed to uh manage our own cost as well as um you know as a startup

and we looked for solutions and we started using open cost and we realized it was so valuable we ended up using it in our product. That's how we got into it and then we started contributing uh and then we eventually you know sort of became part of the community and there are lots of um end users as well not just vendors end users as well who actually

contribute maintain plugins etc. So it's an opportunity for you to um become a maintainer and hopefully that's one of the outcomes of this talk that I can at least uh influence a few of you to um drop by the open open source project the open cost project by the Slack channel try it out and may perhaps submit a couple patches um you know um let's see how

it goes. All right. Okay. So uh let's talk about open cost. What is open cost? It's a it's a vendor neutral CNCF project. Uh it's right now um uh incubating. So we went past the sandbox uh stage and um keep an eye on the time and um we you know it provides the value of open cost is at a you know if I say it in a

sentence it provides a cost model uh for kubernetes and it provides two things primarily one is the efficiency of your resource usage number two uh is the cost of u your resource usage and I'll get into the details in the in the next couple slides. It has multicluster, multi cloud support. Um, so AWS, Azure, GCP. Um, if you're running on prem, you can give a custom pricing

sheet. You're welcome to add support for the other clouds as well. Um, it's um, I would call it a straightforward integration, but uh, you and actually we are doing something right now in Opencast to make it easier for people to add support. Oof. Uh add support for uh other um uh pricing API integrations as well. Has a decent extensible architecture. We'll go over that. Um and a

and a somewhat flexible deployment model. I'm stretching the word there, but um I can guarantee you it's um you know, it's fairly easy to kind of uh get it up and running. So why does this matter? And I think the room is full kind of answers my question a little bit. People, you know, Kubernetes adoption is pretty mature these days and people thinking about day two ops

and cost management uh is an essential part of it. Uh cost visibility uh is an essential part of it. So that's why people are paying attention to this because it's a way for you to uh keep your Kubernetes costs in check. It provides total cost visibility. So your cloud vendor bills will tell you how much you're paying for cluster A, cluster B, cluster C. But open cost

will give you that breakdown to say this is how much you're paying for part A, part B, pod C or container A, container B, container C and then you can aggregate them, right? You can now say deployment A costs X amount of dollars, namespace B cost X amount of dollars. That's what it provides, right? And these unit economics is important because people use that to a just

track spend um you know build things like uh cost alerts, budgets, charge back uh and in general um you know it's it's good to keep your cost down um even if you have a lot of money to burn because some people also care about carbon cost and stuff like that. It keeps your general power consumption down as well. If there's one thing I want you to take

away from this talk, it's these sort of diagrams to kind of understand. Um, if you take your communities cluster, um, I like to talk about the three zones. Uh, one is your overhead costs. Uh, this is like your control plane cost. Um, if you have any licensing, subscription cost, this is where this is a little bit beyond sort of open cost itself, but very applicable. So you

have your overhead cost. Then you have something called the resource allocation cost. This is your compute CPU, GPU, memory, disk. The important part here is when when your nodes get provisioned, especially on the cloud, it starts to uh you're starting to get charged whether you use that or not. So resource allocation cost is at the time of allocation, it starts to charge. Resource usage cost is driven

by activity. So we're talking about network cost egress cross zone cross region um you know um the best I guess what is it called the best uh practice for deploying uh kubernetes on the cloud vendors these days they talk about c cross zone you know multi-AZ but most people don't quite look into the details cross traffic um can be very expensive uh can be super expensive depending

on, you know, if you have very chatty applications between the two, right? Um, some cloud vendors charge coming in and going out of the of a of a availability zone, right? Now, open cost uh before we get too excited um the open cost doesn't calculate network cost out of the box, but um the data structures are there and that's where some of the vendor products come in.

I think we will eventually work together to probably add network cost in the open source project as well. Uh but the data structure is all already there. So that's your cluster cost makeup. Uh the next one I want to u point out is the cost of a container. That's a smallest unit right. Um again you have allocation cost and um egress. Um I forgot to add one

slide but um what is kind of important to understand is if there's enough time I'll kind of try to pull it up. When you when you think about the cost of a container um there's two things. One is um how much you requested versus how much you're actually using. So let's say your request and that's a slide that's missing so I'll try to explain. Let's say you

requested one CPU and one gigabyte, but you're using only a quarter of a CPU and um you know half a gigabyte. Guess what? You're still paying for that. So when we calculate the cost in open cost, it's request or usage whichever is higher. So because you're paying for request the moment you request um the pod won't get scheduled if Kubernetes can't give it to you. if it

can give it to you, if you have autoscaling, it'll autoscale as well. So the moment it gets scheduled, you're incurring a cost. The second example is you've given it one CPU, 1 GB, but you're using 1.25 CPUs and you know 1 1.5 GB this time you're using more. So that's what the cost of running that container. Again, request And now based on that you can calculate efficiency

metrics. You'd be surprised uh to see if you that's the first that's the first time when I deployed I got this um sort of you know um bit of a shock to see oh some pods were grossly oversized right so you you get that visibility right off the bat where you can see the efficiency and sometimes you will see some pods have more than 100% efficiency it's

not a bug it's because it's using more than it requested, right? And based on the um the cost of the container, now you can aggregate it, right? You can calculate the cost of a pod, deployment, namespace, etc. And then you can keep rolling this up into business units, etc. So now you can build chargeback based off that. Any questions on this? No. Good. All right. So let's

move forward from here. Um hopefully everybody understands this one right. So next one is uh we get this question asked almost every time someone comes to the booth uh the open cost booth that is what's the difference between open cost and cube cost. So a little bit of history, cube cost donated open cost um as a CNCF project and open cost remains like the vend neutral standard

um and it's it's an unbiased integration vendors like CubeCost and Randoli uh like us we build on top of that um you know we provide the the rest of like you know like um the rest of the sort of the features on top of it but there's plenty of end user customers that have built stuff on top of open cost as well. Um there's also some of

the cloud vendors that are using open cost as well. So um it's a decent ecosystem and uh I invite all of you to join. Uh it's a brief history of it. In the interest of time, I'm going to sort of move forward here. Um you just um um you can Google this. All right. All right. So let's go to the architecture. This is important. Um, open cost

uh if you deploy it, you'll see there's a pod um and um you know the the current implementation um depends on Prometheus. So what essentially happens is just to explain this open cost relies on usage metrics that comes from coupube state metrics and then um it uses the cloud API uh pricing and writes the cloud API pricing uh as a Kubernetes um uh metric as well and

then later on it scrapes all of that and starts to aggregate and then there's a um a nice set of uh APIs uh where you and grab this data and build whatever you want to build on top of that. So in a nutshell you can if you think about it it's grabbing usage metrics grabbing cloud vendor pricing uh if you have on prem it's an it's a

custom pricing sheet you give calculate the write it back to Prometheus and then you can fetch them through um these APIs that helps you aggregate and stuff like that you can aggregate across labels etc all of that stuff that's kind of like the um the basic thing uh and then on top of that you have um a plug-in architecture so people have written other things uh like

like a data dog plugin. Um there's a snowflake plugin that's kind of working through the uh PR pipeline right now. Um and you can you know build stuff on on sort of on top of it as well. So that's kind of like the Um all right. Now what's new in open cost? Um am I clear at the back? Sometimes I tend to drop the mic. You guys

are good. Okay. All right. Um so um just like any open cost project I mean any open source project we've had um you know um difficulties in you know improving test coverage and stuff like that. The the the community has done a great job in bringing integration testing and stuff like that. So we have improved integration testing and this is where um more people can help contribute.

Um we run it we try to run it for every build and there's uh there's also release testing and stuff like that. So there's improved coverage right now. Promless mode. So if if anybody has used a cost in the past, especially on very large clusters with lots of nodes, lots of pods, Prometheus is a bit of a bottleneck, has performance issues. So there has been um um

a requirement from the community to go for a promless mode. Um we've we have an experimental version. So that's again an area where people can contribute. I'll show you what the architecture looks like just like everything else. Now we have an MCP server actually that was contributed by um um someone who did an LSF internship. Um so um you know uh I I I'll actually show you

that part really quickly. So there's opportunity for you to um contribute there as well. And then finally coupube model u this is an important one. Um if anybody use open cost you will notice that the pricing sometimes differ from what your actual billing is. That's because it uses the public pricing right the reconciliation is where some of the vendors come in as well. But we want to

make it easy for everyone. So there's something called coupube model where you hold the resource usage model separates from the the allocation and assets API um so that it allows you to uh do better um you know pricing um calculations. So that's kind of like the the other architectural thing that's been kind of worked out. It's still pretty new and it's been discussed. So that's something that

people can join the project and and influence the direction. All right. Um do you know how much timing I've got? Uh oh yeah. Okay. Good. All right. So um just want to show you the promulous architecture. I think you guys saw with the Prometheus. So here's the difference is there's no sort of Prometheus. We we scrape the matrix and then um there it it's written to desk

um it's a well file again um there is um you know still it's being sort of worked out um you know we'd love for folks to test uh provide feedback patches testing anything and everything is welcome. Uh so um this this is a really good solution if you're hoping for very large clusters with you know tons of workloads um you know running on that cluster. MCP I

think it's probably best shown with a demo. Um hold on see if I can yell because not very good at just doing stuff with one hand. Okay. All right. There you go. So, um I've got a a port for I have a Kubernetes uh sorry, I've got uh uh let me see if the same instance is around. Yeah. So, you know, I've got a re uh readonly

version of open cost. >> Was that >> Oh, yeah. That's because I just put it down so I could type. Sorry about that. All right. So, I've got OpenCast running and then I've port forward that open cost version. Oh, see I've got lost that one as well. So, I got a readonly version of it. Um, so it doesn't mess up anything. Um, and I've set up goose.

It's one of those um um tools that you can use to um do something with an MCP. I need both hands to type. Um so, I'm going to put this mic down and I'm going to ask it to give me the top five most expensive name spaces. I think I made a spelling >> I think I made a spelling mistake, too. Ah, it's fine. Oh, >> thank

you. >> Give me 30 seconds to switch. >> No worries. >> Okay. >> Yeah, I'm good. Yeah. Oh, he did, I think. Nope. Nope. Okay, we come back. All right. So, um let's talk about plugins until the MCP go responds. Uh this allows you to bring in other costs. So a lot of people ask all sorts of questions about you know can we bring in other costs

into this. So that's why the plug-in architecture exists. Um it's not perfect but it's um it's you know it's something that can evolve into something good. U there is a there's a couple of um um plugins that are already out there like I mentioned there's one for there's one for data dog one for openai uh snowflake something someone contributed to it. Uh this is where we're looking

for a lot of maintainers. So if you're interested um you know it's a good way to roll your sleeves up and get into an open source project. Um the the vision around plugins was to keep open cost pretty slim just focus on Kubernetes cost so there's no bloat and then you know let people layer things in. Yeah. So for example Chennai token cost that's something we ended

up building using a plug-in architecture on top of my cost is a nice Okay. Oh, we got a we got a response here. So, yeah, see it just um you know, it's using the open cost APIs and um tells us you know some some numbers um you know uh it's all based on the really nice API um model that open cost has. So, it's actually behind the

scenes it invoked the open source APIs um to to uh sorry the open cost APIs to get that number. So you can play around with it. Um to um little bit of elbow grease, but you'll you'll get it going. Uh send us a message on the Slack message. Um hopefully this gives you a bit of a taste into Opencast as well. And uh let me see if

I have Opencast running so that I could uh soon after the the UI is horrible. Um we got a I we got to be honest, right? Um, we we got someone contributing right now to kind of make it better. But please don't get discouraged by the the horrible UI. It looks like so I don't know 1980s. Um, it's the API. The APIs are super powerful. It's that's

the value of open cost to me. It's the model and the APIs. Um, it's also if you deploy open cost, it's on a per cluster basis. If you have, I don't know, 20 clusters, you're not going to log into each cluster and look at the UI, right? you're going to use the APIs and grab the data. So that's the value. So do don't look at the open

cost UI and get upset. All right, coupe models. This is the other one. So if you look at like open cost right now, it's getting public pricing and it's a little bit of a pain to do reconciliation afterwards. So what coup model is doing is defer the pricing um um you know calculating the pricing to later on in the pipeline. So coup model is a more simplified

it doesn't have any of the pricing data. It only has um the usage data and then later on the augmented model u we're trying to get the pricing and then support the assets and allocation API. This is what it looks like in code right just just in case. Uh so see the on the left hand side like it's a really big data structure it's cost adjustments network

cost the whole shebang and over here you'll see that that's the new coup model on your right hand side um you know uh it only has usage information compared to the uh left hand side which also has pricing information. So with that clean separation allows more flexibility in calculating a more accurate pricing because sometimes um if you use spot nodes you would realize at the time sometimes

we make the calculation the spot pricing has changed at the time it's actually get scheduled and used right so there are little little nuances all right um oh nope I don't want to do that and uh yeah um oh final thing um we've paid some attention to supply chain security. Um so there's um CI scanning um there's an SMS generation as well. Um again an area where

people can come and support us. Uh we're trying to do as best as we can even though it's an open source project to you know pay attention to security and um you know have these things ready because when people are adopting this in the enterprise the you know these things are required. All right. And then just putting everything together. Um, you know, promis mode, we really need

some help. Uh, you know, plug-in ownership is another one. U you know, it's a good way to get started. It's a very, you know, easy um, you know, just a confined area for you to get started. And then the Kubal iterations, we're very very much in the early discussion stage over here. We need to firm it up. We have um um bi-weekly uh community meetings. Um you

know it's um you know anyone can sign up. One is more west coast friendly. The other one after some discussions we moved it to early in East Coast so people in Asia etc. Europe can uh participate because the other one's at 4 to 5 Eastern which is pretty late for folks in Europe or Asia. So we're trying our best to make it more uh easy for people

to to participate. Um yeah and there are some discussions on LLM token cost and stuff like that as well. There's a proposal right now. Uh we're trying to figure out um how to do it alongside um open telemetry uh and and other standardization effect um you know efforts around this. So that's another area I already mentioned about the UI revamp. Sorry I couldn't get it going but

I'm that looks a lot better than what we have. So if you come to the booth you you'll see the the 1980s Tetris style uh UI. Uh but like I said the value is in the APIs. Yeah. And then it's a it's a it's an open source project. So you know help us write the the road map you know what it looks like you know in the

second half of the year or next year. um you know it's a very u inclusive project like I said like anything else in CNCF we'd hope to have some of you there um and hopefully all of you use it and even a bug report is helpful and um I think most people that's how they get into open source submitting a bug report um you know submitting a

small patch all all of those things help it's also it's a little bit easy now with I guess chai so you So uh uh we look forward to more contributions. Um I am happy to take any questions or are we out of time about just about? >> Oh yeah okay question. Yeah go ahead. >> Uh just in case we're missing something now um if I if we

right now use the current model without cost data I we end up with something like the cube model 2.0 Oh, right. It's just allocation of the name spaces and resources without >> um No, no. So, right now, uh we have no choice. The as soon as we look at the usage data, it's written again, it's written back to Prometheus with the uh with the um pricing information.

It's too tightly coupled, but we're trying to clean that up. Yes, that's the coup model. Okay. >> Changes. Yes, it's a good question. >> No worries. Any other questions? Yes sir. >> Uh yes. Uh you showed that the costs were in float 64s. Have you experienced problems with counting up costs multiplying because of the floating point precision? >> Everybody asked a tough question. No. Um so um

I I personally haven't um but that's something that I can ask the I haven't seen a discussion on the list yet but that's a good question to ask on you know on on the Slack channel. I haven't heard of um I haven't seen anybody complain yet about it, but that could happen, right? It's Yeah. Anyone else? Yes, sir. >> Ah, yes. Um yes. So, there is a

um unfortunately I'm the wrong person to ask that question, but there is um there is support for the focus pack. Yes, there's a pocket exporter as well. So for folks who need it in a different format from an ex yeah from data export perspective as well any other questions uh yeah one more >> are you interested in infrastructure outside of kubernetes >> uh great question I think

we frequently get that I think that's a good um question to pose in you know in the community slack channel um at least I know about genai cost but There's still been a little bit of debate, you know, whether it's just going to be strictly on workloads and uh LLMs running inside Kubernetes or can you look at it outside. Um there is a cloud cost part to

Kubernet uh in open cost which supposedly can bring in things but it's still you know I wouldn't say it's complete. So if you want to look at like your VM costs and other things is a little bit hard to do that. Yeah, that's a good question. Oh yeah. >> So from what I understood, you basically pick up the pricing from like the public pricing. You don't connect

to the internal uh billing dashboard, >> right? So that's yeah um you can override that with open cost. That's what the the vendors do and when the coup model thing will make it more easier for people to do it. >> So that's the other vendors but not in the open. Not in the open source but with coup model separation argue it will make it easy for anyone

everyone to kind of do it. Yes. All right. Thanks a lot. I I really appreciate this full heart. Thank you.