How Statistical Offices Move to Cloud Native Technology - Frédéric Comte & Trygve Tatsuya Falch

This talk discusses the transition of national statistical offices (NSOs) to cloud-based solutions, focusing on the experiences of Statistics Norway and a French counterpart. The speakers highlight the challenges faced by NSOs in balancing security and innovation, particularly in adopting cloud-native technologies like Kubernetes. They emphasize the need for modern infrastructures that allow flexibility for data scientists who often operate outside traditional development roles. Specific tools and frameworks are introduced, such as Onyxia for creating data labs and NICE for application infrastructure. The use of containers and object storage is discussed as a way to facilitate scalable and efficient data management. Ultimately, the presentation outlines a vision for rethinking the data science platform to enhance collaboration and improve the quality of statistical work.

Uh we're going to talk a little bit about how statistical offices move to the cloud. Uh my name is TBA. I'm the chief architect at statistics Norway. Uh and this is >> and I'm Frederick K. I'm the lead of an innovation team at a French national statistical office and I'm currently leading two open project. One is Onyxia, an open source software to build data lab on top

of Kubernetes and the other project is a an available instance online widely accessible to students um researcher public servant civil servants uh in Europe. So national statistical office are public institutions responsible for producing official statistics about population, economy, society. So our mission is to collect, process and analyze data to provide realable and impartial statistics that support public decision making, research and democratic debates. So in this fast

evolving world, we must continuously adapt our methods and tools to answer high quality statistics. As you may have were heard, official statistics are increasingly contested, presenting new challenges. At the same time, AI is worshiing how information is consumed. It's another challenge we must face. Despite this, NSO uphold strict principle of independence, confidentiality and methodological regor to ensure statistics remain trustworthy and comparable. So that's why we are

Eric trigger together to show and to share how we address these challenges in the Kubernetes landscape enabling secure cloud native solution for the future of official statistics. we as a national statistical office face a fundamental institutional bottleneck. It's the tension between protecting systems and data while also enabling innovation in many organizations. Even basic needs like a server, a database, access rights are handled through tickets and approval

changes chains that creates uh what I would call a ticket culture. Uh something that should take minutes, can take days or even weeks. At the same time, it is put in the role of gatekeeper. And to be fair, there are good reasons for that. It is responsible for security, compliance, stability and cost control. The problem is that this often leads to centralized processes that designed for protection

rather than speed. So the result is that innovation becomes a managed process instead of a natural flow. Teams do less trial and error, iteration slows down and exploratory work becomes harder than it should be. This is exactly the institutional tension that cloudnative and self-service platforms are meant to reduce. So today uh let's not be too shy. We let's honestly show you all our walls. So actually we

are quite lucky. Most national statistical offices uh still invest in it. We haven't outsourced it. uh so these wall are mostly internal uh which makes them easier to address maybe uh and that's exactly what we are trying to do. So we will talk a bit about how statistic Norway handle the classical wall between developer development and operation. But in NSOS we have a specific challenge. We also

have strong development skills uh inside business unit because it's our business. um some for example some data scientists are perfectly capable of building machine learning APIs in Python. So let me give you a first concrete story. Uh we had maybe 10 years ago uh a bit less we had our first machine learning API for classification task before the raise of generative AI. So the data scientist built

it in Python using fast text a library from Facebook for natural language processing. But when it came to production IT developers said no no no we need to rewrite this in Java. So they looked for a solution and found a wrapper GFS text. So here we are. We duplicated the work and the artifact was already outdated before it was even used. So at Statistics Norway, we have

modernized the infrastructure by moving to Kubernetes and cloudnative technology from onremise centralized legacy systems. But then we hit the first wall because modern infrastructure is not the same as a finished or usable platform. Developers were still frustrated and by trying to build too much ourselves, we were already creating new legacy. This is why NICE became important for us. NICE stands for NAV's application infrastructure service. It's built

and maintained by the Norwegian Labor and Welfare Administration. They've had talks here earlier uh I know they had last year I think and it's open source and the idea is simple. Teams should be able to build and run software safely without having to fight the platform. So nice provides readymade building blocks secure runtime observability identity and data services and other capabilities teams need in practice. Not just

raw Kubernetes but a real developer platform on top of it. And for us this was also an important strategic lesson. We do not need to build everything ourselves. We can lean on capabilities developed by other public agencies in Norway that meant NAV and NICE in internationally. We see the same pattern in our collaboration with the French statistical office around Onixia which we will talk a little bit

about later. So as I told you in NSO data scientists are often the real developers but just within the title. To give you another example, we even built a dedicated language for statisticians. They would write the business logic in pseudo code and it would be automatically translated into Java obviously. Uh that is precisely the approach we want to move away from today. So as I I told

you we are not in a standard developer scenario. Data scientists are evolving in a fast and complex ecosystem nowadays. In the past, they had only one tool to manage. We used SAS, a popular language and it was fitting all your all our needs. Uh but now with the rise of big data, machine learning, generative AI, they're navigating in a wide landscape mainly open source. Um and we

have to manage the coexistence of all these tools uh at a scale. Secondly, they are not traditional developers. They sit in business units. They are even further from operations and unlike developers, they don't always get the same level access uh to our data or our computing resources. But the key point for what we call um internally the internal data science platform uh and not developer platform it's

the last two points. So you have to know that data scientists always work with the real data not mock data like So they need real data often sensitive which means that they can't just on code on their laptop. They we need to provide them a centralized platform uh that fits their their need um securely. And the last point um a lot of code um isn't made for

long running application. Sometime is just one of code for analysis or exploration. It doesn't always go into production. So all of this means that we have to rethink uh the data science experience and how we create platform that really feeds the reality. containers first. In the past sharing not just data but how we run software was difficult. Containers changed that. They give us a portable interoperable way

to package tools like Jupiter and R Studio and almost everything else. And they keep even one-off work close to production standards. If something useful emerges, it is already on the right path, not stuck on someone's And the next is object storage. National statistical offices work with large mostly static data sets. And object storage fits that well. It scales, gives data scientists easy access and works especially well

with formats like parquet which is efficient for analytics. It is also simpler and cheaper to manage than traditional databases. And this is actually something that we see in our office that the statisticians use less and less databases. they are very content with using uh things like paret and together containers and object storage gives us a flexible scalable foundation for both exploration and and we also keep secret

management simple. Users sometimes need credentials for external APIs and services and we use vault for that. It is not something we treat as a major strategic investment just a practical way to manage secrets safely and consistently. Finally, the data warehouse and metas store. We use iceberg as a modern table format exposed through REST API that gives us flexibility in how data is managed and queried. The result

is a more open and accessible warehouse making it easier for both internal teams and external partners to work with. Yes. So our architecture is not built for one tool but for an ecosystem. For example, Doug Debb is now our first class citizen for um analysis, but we also may run spark or trino from some some other workloads. And as this ecosystem grows, the real challenge becomes a

user experience. So how do we make all this complexity usable for data scientists and innovators in our organization? So when we designed the UIX we had to make a choice introduce new abstraction layer or focus on guidance. We choose guidance because when you create new abstraction people end up learning internal tools instead of standard technologies and that's create lockin and limit portability of skills. With guidance, we

keep the standard tools, but we help the user navigate the complexity. So, Onyxia is simple. It's only our UIX, a simple user interface that focus on transparency and reversibility. You may see on the screen that we provide an emulated terminal to show a user how a user could have done the same action with a bash. So, it's our philosophy. We don't want to be locked into a

maze of custom solution and abstraction. So how we did it? Uh we use a single uh page um that uh connect to every resource server the data scientists want to use. So containers, object storage, etc. And it's unified by an uh open ID connect as a unified access layer. And for example on S3 object store there is some exchange endpoint so you can come with a GWT

and get your temporary session keys and this is exactly how Onyxia work it's exchange token to resource server and then provide a unified experience. Uh for this packaging system our user need to start Jupiter air studio etc. We just provide M charts and we use the value schema on file to build a form based on the constant of this CHMA. We also have some uh specific injection

because as we know the keys of the user for for S3 for example we can fill it the form automatically uh and provide a good environment for the user. We also use something specific uh not specific but not very well known. It's the lookup function into the M chart. So for example, if a user start posg and then p admin the configuration of the posgr database will

be automatically created inside the pia admin containers because we can discover what uh y service or deployment or stateful set uh is in the name space of the user and as our user are used to file explore In Windows for example, we provide S3 browser directly in the web application and also a file explorer powered by data explorer powered by duck deb web assembly was them uh

to show the first few line of a parket file or iceberg open table. So in conclusion, it builds the platform, the statisticians lead the science and together we scale the impact of both. So thank you very much. Uh we can actually show you a little bit of a demo if we have a little bit of time. >> Yes, we have time. >> And maybe if there is

any questions in between, we can just answer them. So what you're looking at here now is welcome screen of one instance of Enixia at uh something called SSB cloud which is a public uh service that uh the French statistical office provide. And on the left side here uh uh you can configure your accounts but you also have uh the service catalog which provides you a lot of

services that you can just start uh including databases, automation different homegrown applications and also data visualizations application. I already have a few services running and as you can see there is also a a quotas system here so you can actually see how much resources you're using. Uh and in this example here I'm just running VS Code uh in the browser. Uh I was just testing out uh

the ins uh own hosted LLM uh to make it make myself a little breakout game. So this is uh statistics Norway's instance of Onixia. Uh and this is actually the forward facing uh uh uh system that all statisticians in in our office are using to to create and produce uh their statistics. So in our service catalog we have slightly less services than the than the open one

that the French have and we also don't use S3 as we are actually running everything on Google. so we have uh fixed the data connection part here slightly differently. We use uh fuse and mount the object store directly into the uh images that we uh invoke. So yeah uh I can start a service and you can also and this is also what uh Fred talked about. So

the instead of actually seeing the YAML files, the YL files is rendered into a user interface that makes you able to actually do stuff in here like uh change the number of uh cores or memory uh and so on. Then I can launch it. And since we're a statistical office and are in lack of imagination, the name of the data platform is Dupla for short. >> So

maybe we can go to questions. >> Yes. >> Do you have any question on what we are doing? You talked about the data scientists basically creating software that then couldn't be used because the data scientists are not developers for production. Is there anything that is improved with this platform? Yes, previously our data scientist um develop on Windows server remotely accessible and um now as they are using

directly containers on a Kubernetes cluster they are less far from production but a technical uh tools can't do can't solve your organizational problems but at least the foundation are good now and the tools the underlying component are the same between development and uh >> and in our office uh we also provide uh readym made templating and these type of things and and we have rigorous training of

the statisticians. Uh so we require the statisticians that work on statistics in in our office to actually learn Python. which is uh it's been a quite an undertaking but uh we've managed so we've trained about uh three or 400 statisticians in in Python and uh we keep having uh new courses that we develop and uh we have support teams that help each other uh within the so

I think the the quality of the code is going up uh we have um we monitor it uh and they are responsible for the end to end uh coding or the stat statistical pipelines. >> So how are you launching things on an external service and then you talking to it to launch there? So the question is how we start services on top of Kubernetes. So Onyxia is

basically a M chart launcher and we start the M chart as the user would have done himself with a command line and is accessible with for example an ingress or any kind of access access to your uh services inside Kubernetes. There is no specificity. Maybe one thing we can say uh uh statistic Norway develop um an extension to provide um OEC access to air studio vs code

of this kind of services but only working with sk on the onxia open source software as we don't want to be tied to specific identity provider we just uh open air studio vs code with a a one-time password. >> yeah, so but if you are using keycloak, uh we actually open source that component. So it's available for everyone. Hello, thanks for this presentation. I guess data scientists

are frequently running long running tasks, long running sessions. How do you handle maintenance of clusters, updates of clusters and always unpredictable that can happen? Um so the the the question is about batch programming this kind of thing and longing task uh for >> yes so there is multiple level uh many of them directly use the notebook and uh we need to make them learn the to not

attach the batch to the web session and and go uh without the dependencies to to this terminal. Uh so this is a first layer of response when we have no time to explain more complex things. Sometimes we provide some uh an SSL for example you can start airflow or we use also argo workflow as a batch processing system. So if they have the skills it's it's far

more complex. So it's so not all will go uh in this direction but we we provide and we help them to use this kind of technology and um yes for the all the cleaning cleaning aspect of managing the cluster for example no user stop containers no one. So we we invented we invented some um concept of posing a container. So the M charts know about uh what

a pose is and if the if there is a PVC and uh the user ask for a pose we can just put the replica to zero and then uh it's it will simulate a pose uh and keeping the the data and PVC uh available for for restart And uh basically we run scripts outside the the scripting part are outside the scope of Onixia and we clean based

on the use of of usage. For example, we are very aggressive with the use of people who request GPUs and not use it actively. We pose the container quite soon. for example, it's kind of ru but it's outside the scope of each instance must decide how how he does the the clean the cleanup. >> So in our office obviously we run on on on Google cloud. So

all running services costs a lot of money. Uh so we have a we have a janitor script that actually goes over all the all the containers and and just pause everything. uh some of them are allowed to you know be exempt from the pause list depending on how long running jobs they have basically. >> You said your scientists were working on sensitive data. I imagine that certain

scientists some data and data Onia sort of isolation how do you manage isolation between users? Yes. It's not onixia we manage isolation between so the question is about how we manage the isolation on S3 for example. Um and the response is totally with access policy in S3 protocol. So on just is able to exchange token and based on what the user has uh we will be able

to to do the good uh access. There is no API between or new abstraction between the user and the S3 storage. So it directly consume the storage with his own credentials and own capabilities. And to add add to that in in our office the the whole uh idea of how you access data and these things these are handled basically outside on XIONX is just the user interface

and and the client. But what what we have is since we have automated bucket mounting on on the images you you can actually choose here when you start the service uh which group you are starting the service as. So you can never actually mix the different uh data from the different groups or the different teams. So they're are only available. So when you start this I I

really have to make a choice. So if I'm a uh a developer a dlfellis person or some random phoenix guy uh I can only access the data from for that group. So that's one thing and the other thing is that you need to have a policy in place for how you handle data and steady states of data as the way we do it. So so there is

a whole comprehensive system for the data platform that's not covered by Onyxia as a tool which I think is also one of the nice thing with Onyxia is that it gives you this freedom to actually have these policies as strict or as free as you want. >> Yes. And uh the assume role with web identity is how we exchange token and you assume a role. So for

example if you want to work on very sensitive sensus data you can ask for this role if you are allowed to and you will have access the specific subpass of you of this bucket or anything and you will have not the other access you may have with other role. So the the standard S3 protocol allow you to assume role and get the assess you need to have

for this working session. >> Yes, we have time for a few more questions I think if there is any. >> Oh nice. >> Yes. Um so we have GPUs uh both internally and on the free instance. uh for GPUs we use the concept of we override the schema based on the role. So on SSPO for example we have um a VIP role uh which provide you capability

to start ash 100 GPUs containers and the other people uh can't they only have says a gigabyte of VRAM for for their session. So you can select some uh containers with uh the the Nvidia things directly on the containers and yes you can just work uh obviously on Kubernetes the use of GPUs is not simple because you request it entirely and you can't share it easily or

it's it's going to change and it's changing now but uh for now it's still like that in our cluster Uh so we are very aggressive when we pose when we clean up. We really want that people use the JP request. >> So Fred, I'm obviously not in the VIP group because I can only ask for one. >> No, no, it's not selector tab. >> Good. >> And

you are not VIP neither. >> It's not here. >> Uh >> there was some more questions. >> Yeah. So um you You are very enthusiastic about nice as IDP. Does then replace? >> No. >> So, so nice is is an IDP. It's an application developer platform and uh it's basically made so so there is overlapping functionality obviously but the the application developer platform is basically for creating

custom applications for the platform which is not related to actually on it could be microservices or services it could be uh registers for example in a statistical office context uh Um and the easy thing is you you could call it you know long living services are running on on the IDP and uh onia is basically a user interface for the statisticians. I may add something about that.

It's in statistical office the the frontier between self-service and uh production is very is a continuum. Sometime at INS we want people to for example publish a shiny application. It's some web application for for air. Statisticians love this uh this thing. We don't uh but they love uh and we are happy if these things uh stay on the self-service cluster instead of the the production line which

is more complex and less accessible to to statisticians. So if an application like a shiny app has five users, okay, you can run it on the safe service cluster and it's not a big deal. Another question. So the shiny stuff, so do you handle less strict cards for that? Uh for example, our shiny decisions love to use that of course, but um we we face a lot

of difficulty having them cloud native 15 factors 12 factors stuff like that. So if you are putting them on so you might not have so much trouble from them getting them to cloud native. >> So a shiny app you can package as a container and then you can run it on an Alexia that's easy. So so we we've done that uh internally. >> Yes we do the

same. No, no, it's okay if they run this. Uh, but shiny, if you want to have multi-user in shiny, you need something like uh shiny proxy, for example, to spawn one container for one user, which >> quite costly. Uh, but if you it's it's a bit a single thread. So,