Cloud Native at the Far(m) Edge: Running Kubernetes and AI... Mauro Morales & Jordan Karapanagiotis

In this talk, the speakers discuss the intersection of cloud-native technologies and precision agriculture, specifically how they utilize Kubernetes and AI in farming at the edge. They introduce key concepts related to precision farming, emphasizing the significant waste in crop production due to spoilage and other factors. The application being showcased is called Tree Scout, an embedded device that employs computer vision to optimize the fruit growth process by creating blossom maps. Additionally, they explain the challenges of operating in the agricultural environment, such as data bandwidth constraints and the need for real-time processing. The speakers highlight their collaboration with the Chyros project, which provides immutable operating systems compatible with Kubernetes for easier edge management. The discussion emphasizes the importance of these technologies in achieving sustainability goals, reducing crop waste, and improving operational efficiency.

Hello everyone. Thanks for joining us. How is your CubeCon going so far? I hope uh it's a good one. >> Awesome. Awesome. That's period. Uh so today we're going to be talking about cloud native at the farm edge. Uh you can credit that bad joke to me. Thank you. Uh it's an interesting topic for me because we're going to be talking about Kubernetes AI tractors. We're going

to be talking about immutable systems. Um, so, uh, I find it very exciting. If you like it, it was because of me. If you didn't, it was because of Jordan. No, I'm just kidding. So, okay. Uh, why don't you get started, Jordan? >> Sure. Thanks, Moto. Uh, yeah, before, uh, diving into AI and Kubernetes, uh, on the edge and on tractors. Uh, I'm going to introduce a

little bit what is, uh, precision farming and why do we do it. Uh, yeah, starting with some statistics. Uh basically it's estimated that uh about 25% of uh global crop production and that's a moderate estimate uh is uh affected by uh posth harvest losses and the reasons are usually spoilage uh labor shortage or uh size and shape irregularities of the fruits. Uh now to give you some

context in the Netherlands uh the average farm is producing about 10 million fruits per year. Uh that's usually apples and pears and if you think about it 2 and a half million um either don't make it to the market at competitive prices uh or don't make it to the market at all and most of it ends up in waste. Um at Ara Imaging uh we are trying

to make a more sustainable uh orchard uh for the farmer for the environment and uh to reduce this waste. And how do we do it? Uh we use common techniques used by aronomists um there are quite uh quite some techniques there but um a common one and it's part of our core application is called uh blossom thinning or uh precision thinning. Uh how it works, you basically

uh spray your orchards with um a salty solution early on in the blossom season, and that causes some of the flowers to dry out. And then the tree uh uses its energy to the remaining uh flowers, causing um less fruit to grow, but larger in size, more uniform in size, and often of better quality. In terms of our application, this is how it looks like. First, the

tractor is driving in the field. We are doing computer vision uh in real time. We are detecting blossoms on the trees. We create a blossom map. Basically, a map showing the flower level of each of every tree and we give that to the grower. Then the grower validates this map, adjusts some thresholds and loads it to their sprayer implement. Uh and then they drive again with their

sprayer um spraying in an automatic way and that basically increases their guilt at the end of the year, increases the profit and also uh they reduce the amount of spraying chemical that they use because we treat every uh tree independently. Now enough about precision agriculture, more about um the product. Uh our product is called the Tree Scout. It's an embedded device. It's basically a set of camera

sensors. Uh it's a ruggedized Nvidia jet on board integrated with some peripherals. Um integrated with a tractor GPS for uh geoloccating uh trees and blossom on the field and uh also with a universal terminal that goes inside uh the tractor cabin for operating the tree scout. the application if you if I run the video um basically you can see the application uh running when the tractor is

driving in the field. Uh that's a lot of data. It's a large amount of data. We use about uh we use 15 frames per second. Uh resolution is 1080p. So uh about 25 megabytes of raw image data uh and that has to happen in a short uh amount of time. Uh it's crucial that um the spring happens during blossom season before the flowers uh dry out. And

uh apart from that we also are limited with bandwidth. Uh tractors don't come with Wi-Fi on board yet. So we use 4G and LTE networks. So large amount of data um short time span and limited bandwidth basically calls for an application that does most of the heavy lifting on the uh on the edge uh before sending more lightweight data to the cloud. And that's why we use

GPU enabled devices like uh the Jetson uh to do all the AI inference on the edge. uh we learn we we run machine learning model uh models on the edge and then we send uh more lightweight uh more and of course uh yeah AI on the edge is quite a new domain quite a niche topic which is uh the topic of this presentation as well um and

we're going to talk more about it at this point I'll introduce myself uh I'm Jordan originally from Greece um living in Amsterdam and working for uh area imaging as a software engineer. I mostly focus on uh IoT IoT data um our edge software but also lately uh I've been quite involved into bringing um immutable containerized uh OS to the Tree Scout which I'll give the floor to

Mao to talk more about that. >> Uh thanks Jordan. Uh so yeah my name is Maro Morales. I'm a staff engineer at Spectral Cloud. uh Spectre cloud is doing uh different uh management of uh clusters of Kubernetes on the cloud uh on edge on prem uh basically uh solving the complexity that you might have there. Uh I'm also a maintainer at the uh Chyros project. is a

CNCF sandbox project and um we do basically uh life cycle management of systems uh through image based uh Linux distributions that can be Ubuntu, opens uh all the major ones and we also uh have now our own Linux distribution called Hadron which is uh very minimal uh upstream focused um Linux distribution. And last but not least, I also host the cloudnative uh Belgium or cloudnative Brussels uh

group from the CNCF community. So if anyone is interested to talk uh there also please uh uh contact me. Uh you have my information there. Uh actually the talk is uh going to be uh at the upper level we could say. So we're not going to dive deep into technical aspects but uh afterwards uh we will probably leave some time for questions. You can make obviously uh

technical questions or again you just contact me uh later on. Um so yeah uh running at the edge. Uh can I see some raised hands? How many people are running systems at the edge at the moment? Okay, I cannot see much because of the lights but I saw a few hands going up. Um I'm saw maybe like 50 people or so. Uh I'm sure if we would

sit down and try to come up with a definition of what the edge is, we would come up with 51 definitions of what the edge is. Right? So um that's also part of the problem. There's no one uh solution that fits all. Here I am putting some issues that you might have at the edge. You might encounter one or you might encounter uh all of them at

once. And the problem is that the edge can look like uh you're running in a tractor like uh Arya imaging is doing it. But it can also mean uh you have uh maybe a grocery store and you have thousands of locations across the globe. Um it it can mean so many things. It can be small devices or it can even be a cluster of uh edge uh

specific um machines, right? And uh so let's just uh address one or two of these ones here. One of them is maintenance, right? Like uh you might not want or or it might be too expensive to send an engineer to all of these locations or maybe it's in a very very remote location where sending someone would be uh dangerous or uh extremely expensive, right? Uh another thing

is sometimes you have to make do with what you have um at the edge. So maybe there is no uh technical person that can help you set up the machine. You want to make some sort of UI that is uh easy for anyone that can uh at least boot the machine and make one or two selections in a uh boot menu for example. Um and uh a

big one of course is security. uh if you are putting if you're exposing your devices out there uh like I was saying in a grocery store for example uh it's the likelihood of that device uh being hacked fix physically attacked all of a sudden uh grows exponentially like it doesn't mean that everyone is going to be an attacker but it means that all of those people could

be an attacker right and you might not want to uh get to that point So uh that's why we came up with Chyros. Chyros is a system that can run it's Linux at the end of the day. So it can run pretty much everywhere where Linux runs on cloud on prem uh on edge. But the difference is that we uh many of the features that we designed

on Chyros uh start on the drawing board uh thinking of the edge problems uh that they can solve because uh if you need it well you use it uh on the cloud as well or on prem uh but you cannot do it uh the other way around right uh so it has to be secure for the edge and then if you need it well you also use

it on your data centers uh so that's exactly uh some of the topics The first one like I was addressing is security. Uh what do we offer for example in that sense. Uh the first one is called trusted boot. Uh trusted boot is a mix of using secure boot which uh many people might already be familiar with. Uh but on top of that we use uh systemd

in order to be able to um measure and attest that the uh kernel the unit ramfs and the kernel command line are the same ones that you uh signed and allowed to run in your system. If for some reason there's a chain uh there's a change there the system will simply not boot. it will uh uh brick itself which might be a very good situation for you

because I don't know uh maybe the the device might be expensive but more important than that maybe the data that you have there is intellectual property or it's uh user data if you're talking about uh data from um related to governments or something like that it might be too risky to expose it right so um that that's one of the uh cases that we try to address.

Another one is uh no configuration drift. So if you have any kind of uh way of provisioning that does uh in place upgrades in place changes, well yeah of course the changes can be for good when it is running uh the way you design it to be. But the fact that a system can change for the good, it also automatically means that it can change for the

bad. So uh for that scenario we have uh certain part of the system that it is immutable which means that uh for example if you try to do an apt install it will simply tell you I'm sorry but I can't because I cannot write on the disk. There is of course areas where you can write because otherwise the system would be completely unusable. Um um you you

have your user data uh or where or places where you can write logs for example. Um and uh last but not least, another aspect uh that Kyros has is to try to do simple maintenance. Like I was saying, you might not have uh the ability to send an engineer to every uh location. So you want to have options in which maybe just from a booting menu you

can reset the machine or you can completely uh uh wipe like uh factory reset the the machine, right? And uh to this regard we also integrated pretty well with Kubernetes because we offer a solution uh Chyros operator that allows you from Kubernetes um define how you want to do the upgrades of of the nodes and from Kubernetes the full machine is going to uh reboot and start

uh with the new uh version of the OS that you want installed there. Um and uh important here is to say that then you have also um uh possibility to do canary uh upgrades uh to decide how many nodes you want to be uh rolling uh at a time for and I think that uh is all I have to say for that part. >> Yeah and actually

uh Kubernetes was the part that the feature of Kyros that stood out for us and we uh decided to go for it. uh to give you a bit of historical background on that. When I joined uh a imaging 3 years ago, uh software architecture on the edge was completely different uh than what it is now. It was a single Python application uh running inside a docker container.

Uh we were using Python's multipprocessing library. Um we were having a watchdog process, but basically um the tree scout is connected to the tractor's battery. So um we don't really have graceful shutdowns. The engine can turn off at any point and we have a hard power cut. Uh so processes can fail. The watchdog also fails. We cannot really watch uh what's happening. Um and that's where we

thought uh yeah we don't really have control over our application. We didn't have um control over the state of our of our app. Um no orchestration at all. So we thought we have to uh change course and um we only had one year until the next blossom season. So we needed a functional product within within that time. Um we thought of the traditional approach that is basically

um embedded uh ROSS robot operating system with uh C++. It also comes with Python nowadays. But the team didn't really have um the expertise for these embedded applications. uh it would be quite a steep learning curve. We would we estimated that would need more than a year, one and a half year probably to um make it happen. Um so we uh we were thinking of hey maybe

we we have to introduce a new container uh message broker container but then we also had to manage this container and that's where Kubernetes came um into the game. Uh coincidentally uh I think it was three years ago at CubeCon uh one colleague ran into a spectrocloud and they told us about K3S which was quite a new project back then. Um so we thought yeah let's let's

give it a try. Um we completely refactored our code. We we split our code base into actual microservices. um deployed our single node cluster um with K3S on the Jetson and we actually had a proof of concept within three or four months. We were surprised how how easy it was to uh make something simple uh for the beginning but then uh we added all the layers that

we needed and honestly it plays quite well until today. Uh is the third uh blossom season that we are uh running with this setup as we speak actually. Um, and we have control. We we can actually troubleshoot. We can identify the root cause of the issue and we can easily deploy uh a bug fix into into the cluster into our application. But of course, we're not pioneers

in uh K3s. Uh I think there are other use cases as well. >> Indeed. uh so of course uh you might want to be reassured whether uh other people are also uh running these kind of uh setups. I'm not going to go into detail but you can scan those QR codes and you can see two different talks uh in cubecon one uh with general electric healthcare and

how they are running uh a system with pallet from spectral cloud underneath is using kyas and uh the other one is uh densely uh sedona they uh ship dental devices all over the world uh both of them I think are very interesting and uh the funny thing is uh that was the talk uh specifically that the guys from Arya uh joined and how the whole conversation started.

So who knows maybe this is the talk that starts the conversation just come and like I said please uh reach out um indeed. So if you're interested, please have a look at those. Uh yeah, in the end, uh we we did end up uh adopting Chyros into into the Tree Scout as the main operating system and uh that um brought quite some benefits for the company um

for our operations mostly. Um last year uh this time of the year we had about 25 devices uh deployed in the in mostly in uh uh Europe. This year we have about 60 also some in the US. So our fleet is um is growing quite fast. So uh a homogeneous fleet was was key into what we are doing. Basically um we are heavily dependent on the Nvidia

uh stack the Nvidia ecosystem as we use Jetsons and this is a rapidly growing uh ecosystem that if you want to keep up to date with the industry standards you always have to to upgrade stuff that uh live in your host uh like uh uh CUDA the GPU interface of Nvidia uh Nvidia Jetson uh sorry Nvidia Jetack uh things like that. Um but also we had the

the need to uh the need for upgrade uh our firmware on the host for example for cameras or some some Nvidia patches or for third party hardware for modems. Um and it could also be that we want to upgrade uh to to to update a network setting. uh all these all these things don't live on the application level but on the host and Kyros provided the uh

sophisticated way to do this overthe-air upgrades on a system level. Um how did we benefit from that? Basically uh we don't have to send uh field engineers as Mao mentioned we don't have to send people in the field to do the actual upgrade. We don't have to send someone with a laptop and a cable to flash devices. We just do it once. um from our uh office

basically and then uh we're done. We we use um uh overtheair upgrades for anything uh host related. Uh in terms of operation, it was also quite useful uh that now we we have everything defined uh in a docker file. Our whole uh OS structure is defined there. Uh so we know uh what to expect from our fleet. we can write reliable troubleshooting guides for uh our operations

and then we don't really need uh to involve engineers for um this kind of support uh when something goes wrong. We have very standard steps into solving problems because we know uh exactly the package versions, the installer versions that are deployed across uh our field and even if someone uh would try by mistake to install something, they wouldn't be able because uh the system is red only.

Um so yeah, easier operations also in terms of provisioning. um we have to provision our devices uh from our uh office location because before sending it to a customer and with Kyos basically we have everything defined uh in the OS image. uh we can actually send this uh OS image to our um to our board supplier and they do the flashing for us and then the only

thing we have to do is provide device specific um certificates or resources uh names uh which is quite minimal and then we have the device ready. So this really uh shortened our provisioning time uh to make a device customer uh ready uh but also for uh for our customers for the growers um it seems that they have benefited from from this transition. We are aiming for a

20% reduction in uh crop waste and uh even though that's quite difficult to measure especially since we are operating for uh less than three years with paying customers uh it seems that uh the tree scout and the product is widely adopted uh mostly here in the Netherlands and northwestern Europe but is um but is being uh is being spread out and um all these wouldn't have been

possible without us trying daring technologies like K3s, Kyros uh and uh reaching out to the open source community. >> Awesome. You cannot imagine how exciting it is for someone uh maintainer working on a technical project to know that people are actually using it and using it for interesting topics uh like like this one. There's nothing sadder than having a cool solution but nobody adopts it, right? So,

uh this is for me the first uh big win. Uh it's it's very good to hear that um kyros works so well for area imaging. uh but uh on top of that we have uh these three different areas where the relationship between uh um CNCF project and a company uh benefited also the um open source project. The first one is that uh as you can imagine being

a open-source project we don't really have the resources to go buying those fancy devices uh that they are sending to their customers. Uh and u the fact that they can do all the testing for us uh means that uh they can give us feedback we can do changes into our core uh project and make their lives easier. uh I don't know the exact numbers but let's imagine

uh something like uh if their original docker fire was something like 150 lines it was uh reduced to something like 30 lines just because a lot of those parts were moved into uh one component we call chyros init which is in charge of transforming that image into a chyros image um and that also means that whoever comes now with I think it's the uh orin nx that

you guys are using so now anyone new that uh starts um that wants to try cyros on a or in an X well they have all those uh benefits right like they they can uh have this oneliner that converts the their images uh the second one is uh feedback on using kyros uh overall when you are in an open source project uh one of the hardest parts

is to hear how things are going I mean when they are not going well people generally do approach you but when they are going well is very hard to hear from them. And uh in this case, having a good relationship with them has really benefited because uh we understand obviously where we need to make changes, but also where it's actually uh useful for them, where it's clear

and uh which parts we should continue uh doing as a open source project. And last but not least, uh it has improved our collaboration mechanism. Uh what do I mean with this? Um as an open source project you you have a certain type of governance right like some people are maintainers other people are contributors etc etc but uh just having the uh project open uh is not

enough. We want to have uh records of everything. If someone starts coming as a collaborator we want to be able to um put it out there that it comes to voting mechanism. And I'm more than happy to say that uh part of this collaboration part of uh William Resour joining as a maintainer uh has uh forced us into okay we need to do this now so that

we can properly invite people uh to become contributors so that we can give them also their recognition because of all the work that they are doing. At the end of the day they are part of the community and everything is open everything is uh um can be attested uh right um that everyone can see it the full community can see it and here you can see a

demo uh of the end result basically after we collected the data we processed the data we made a blossom map this map was loaded to the sprayer implement of the grower and yeah basically this is uh the end result saving saving uh fruits uh reducing waste and uh reducing spraying chemicals. Thank you and we are curious for your Thank you very much. Uh the first link is

the carros documentation. The second one is a blog post that we did together uh talking about exactly more or less what we talked about today.