Room 101 Friday Mar. 06 - SCaLE 23x

This talk focuses on the role of NixOS in managing edge network infrastructures, emphasizing how its configuration management can streamline operations. The speaker, Craig Jackson, discusses the significance of establishing reliable edge networks to ensure low latency and compliance with data sovereignty regulations. He highlights the challenges faced in setting up these networks, such as managing configuration drift and multiple tools like Terraform and Ansible. Craig shares his personal journey of adopting NixOS and how it solves problems related to deployment speed and consistency. He explains how NixOS has simplified the management of single-use machines in edge scenarios and transformed his approach to deploying Kubernetes worker nodes effectively. By integrating NixOS into their workflows, Craig illustrates the potential for improved operational efficiency in edge computing.

Hello. Hello friends. We will be starting in a few minutes. So, I'm hoping the people outside can hear me. Um, but if you can start filtering in and getting settled in, we'll be opening in about three minutes. So, welcome to the first edition in this format of a >> Hello. Hello. One, two, three. This is a mic check. >> Keep in mind that we have a fedor

>> booth downstairs on the expo hall. We are be just let us know. We will find the person you need um and arrange for time for you to ask questions. But thank you. This is getting started in open source and Fedora. I'll just give everyone maybe 30 to 60 more seconds to filter in, settle down, and then we will kick off for today. Are you roles they

need contribute? >> all right. All right. All right. Hello friends and welcome to day two of Planet Nyx. Um, yeah. How's everyone doing so far? Have we been having fun? We learning things? We meeting new people. Yeah, I like the front row enthusiasm. Love it. Um, so a couple just standard updates today. We will still have tracks in both room. Well, we'll have talks in tracks one

and two. So, both room in one and two. There have been a couple small schedule changes. So, do check the online schedule. That is the one that's going to be the most uptodate. Um most notably we have swapped the tectonic section with the steering um se section session >> and also ilco's wom talk will be getting moved into here on the main track. >> Yeah no problem

>> um >> it's funny we were just joking about eventual consistency and everyone filtering in and being ready to go and that's just how it goes sometimes when you're live. But otherwise, hopefully everyone's having a good time. We've really enjoyed hearing the conversations that are happening over in the main floor. People have been talking about the things that they've been building, how the community has grown since

like the last 20 years of different either online or in events, and I think this is the third or second Planet Nix second. Yeah. Oh, I've been to both. Cool. >> Ah, Nickcon. I knew there was a catch. I was like I think it's a two or three thing or >> but yeah so we will be having the workshops in the other room. We'll have sessions here

as I mentioned. Um lunch will be similar to yesterday. You will have a 90minut break to go get lunch and then come back. You know we'd love to have you come back. Otherwise we will still be doing video interviews out there. It'll be me. So if you see my sparkly sequin jacket somewhere out there with camera, feel free to come say hi. Talk to us about NYX.

I didn't bring it up here, but you will get a special branded little moonflake cereal box for part partaking most open source projects. >> I'm ready to introduce Ron, but that's okay. We've got a couple minutes. >> Um, OpenStack for >> we are getting a second audio feed. >> We have our own system. We then mirror to GitHub, but if you're going to make contractions, it's somewhere

else. Um, identify your interests and your skills. >> yeah, this is a new one. Do we have any AV friends in here >> from the AV team? >> It's gone now. >> And now I feel like I'm hearing the voices inside of the room. >> Is that speaker? >> Um, but look for projects with good >> docu >> because if there's no documentation, >> I think we're

getting a double track on audio. We're hearing somebody else. >> Actively with the >> I love this. We've got front row helping troubleshoot for all the people in the back that can't quite hear what's going on. And I'm kind of really enjoying the spirit of camaraderie over here. >> I know. We were joking right before we went live. We're like, there's always something that goes wrong. It's

just the nature of live events and tech. >> Yeah, I've got to enjoy the fun, right? Keeps life fun, >> keeps it interesting. >> yeah, this is this is 101 >> access type of cont. >> Hey, that's me. >> You don't need to know all the pieces that connect be like a one repo friendly. Um, look for Right. So, thank you everyone for attending and we're going

to get started with the opening ceremonies with Ron and he will be talking about a secure ny the state of the union. We are doing it live. There's a button here, but it didn't help it. So, maybe we need to wake up the projector. I think if we just frontload all the troubleshooting, the rest of the day has to go smoothly, right? >> It's turning on. and

double check. And if it's not coming up, toggle it. There we go. There we go. Y Okay. Almost ready. So, it was dimmer yesterday. We can I think you said that too, right? We could do some more dimming. How's it looking in the back? Can we see it? Okay. Yeah. Thank you. >> Doing it live. All right. Here's Ron. >> All right. Sorry for the delay. Thank

you everybody for joining us on Planet Nicks day two. We're kicking off another day of a lot of talks, a lot of energy. I know it's 10 a.m. and probably we all went to sleep at 8 and woke up at 6 and ran a half marathon and I'm just giving you um pretty much my my friend Ben's uh schedule uh for those that know. But anyways, um

thank you all for coming. To kick off, what I like to do is uh anybody heard me ever talk about the types of fun? Anybody? Anybody? Phillip. Great. Two people, Tom. Um, so I I like to consider things in life in in three types of fun. Uh, someone introduced me to this about four years ago. And the way I like to think about it is pretty much

that, right? Type one fun is it's like fun to do, fun to remember, uh, just fun end to end. Type two is kind of like, you know, it's a little hard. It's kind of like working out. It's not maybe the funnest to do, but you get that immediate rush right after, and it's also fun in general. Um, and then type three is kind of getting chased by

a bear. Um, it's not fun to do. It's not necessarily fun to remember, but it makes for a great story. And honestly, that's how I think about um organizing Nyx related events. Uh, I'm kidding. It's it's definitely type two, but um all I'm saying is that a lot of folks put in a lot of effort. Um, and I'll and I'll call them out in one sec. So,

I want you to remember that when uh you're thinking about organizing a NYX event. But, um, here's me. Here's me smiling. Here's me smiling exactly six months ago. Um, this is Ron when he was naive. Thank you. This is Ron when he was naive. Uh, how many how many parents do we have in in the crowd here today? Um, I don't know how you do it. Um,

this is when I told the community that I'm I'm expecting a kid and and you know there's that kid now. Uh, so it was it was my wife and I were expecting and then that kid happened and then uh he's three months old now and he's here with us. Um, I don't know. Kudos to you guys. Like >> yeah, we're we're we're paused right now. Um, so

anyways, um, lots of lack of sleep, so sorry for the for the lack of energy. And again, if you want to remember how I smile, that's how I look like. Uh, I'm kidding. It's it's good. but quick shout out. A lot of folks were involved in making this happen. Um, I don't know if any of you are here in this room. If you can raise a hand

and keep it up for just a second, Dan. Uh, just making sure you're listening. Raise your hand. Um, anyone else? Anyone else? Wow, they're all outside. Oh, there's Rock. They're all outside still organizing Planet Nix. Uh, there's a bunch of volunteers. Justin's around here as well. Uh, I want to take a pause and just and just thank everybody that made Planet Nix, you know, happen. So thank

you guys. And then obviously there's there's you know last year kind of flocks completely soloed it. This year we we started opening it up to a little bit more of partnerships for for organizations to also come in. So we had a nice group of a handful of folks that came in and and partnered making it happen. So, thank you to them as well and and hopefully we

can keep uh doing the Nicks magic and you know, the next thing you can help fund is Nixon, which uh I think just got announced in another very exotic European location. We keep it right now at Pasadena. Um it is exotic, especially certain times of the day. Um, so how many of you feel that you know how to talk about nyx with everything that's happening in the

industry today? Yeah. Yeah. It's it's pretty it's pretty I'm going to say this only once. I'm not allowed to say this more than once because then it goes from PG to 13 to R. But it's pretty hard. Uh, the way I like to think about Nicks and the way the reason I'm sharing it is because I think that's our role, right? That's the role of the folks

here in this room, the role of the folks listening in, the role of the folks in the ecosystem. It's about how do we continue to establish a sustainable community and an ecosystem that can thrive 5, 10, 20 years, right? And way further along than any of us want to continue putting up with it. Um, and at the end of the day, it comes down to how do

we communicate? So, I've been using this since pre-hat GPT. Um, what I added is that little thing plus a lot. The way I like to think about Nyx, and I hope that's kind of helpful to framing for folks to to consider when they're talking to folks, is that Nyx came around 20ome years ago and and it said there's an order problem. And that's the complexity of software

development. Why? Because every time we reach a new frontier, we're human. We like to take that new frontier and we like to find something new that's harder to do with And that's good. That's great. A lot of times it bring us a lot of cool things, sometimes not so cool things, but usually I like to be optimistic about And software engineering is an order-end problem, right? And

along the history of software engineering, we've tried to solve it with different patches, with different band-aids. Uh I think containers for instance, right? We were just here yesterday with Kelsey High Totower. They have an amazing utilization within orchestration, within deployment. Why? Because taking a box and shipping stuff inside of it works. It's a square. We work in squares. Nvidia works in triangles. I know. Um, excuse me.

Um, but for instance, there are areas there are areas where that box just doesn't work and we ship that container on the left. So, we've added all this patching to deal with that complexity in ways that we didn't have to. And before I go into, you know, the solution, a lot of folks are asking me, does Nyx still matter? Does it still matter in the age of

AI? agents? I think recently I've heard someone on Twitter say that package managers don't matter, right? Package managers don't matter because agents are going to just be like, "What do you want to write? You want to you want to ship a new age CRM? I got you, bro. You need that library. I'm going to write it for you. It's going to be ultra secure." Um, as much

as I love our our model friends, I personally will not trust them to right now package everything for me. there is a lot of uncertainty. But from my perspective, this doesn't change anything about Thank you. Um, this doesn't change anything about Nyx. It just makes the problem more complex. Hence the N plus a lot, which is a mathematically correct equation. It just means that the complexity has

grown, right? There's I think in 2022, uh, GitHub came out with a state of, uh, you know, state of GitHub or whatever they call it and I think they said that there's 22 25 million engineers in the world today, right? That's where the complexity was. if you use that as a metric. But now I would say that each engineer has multiple agents running. And I think in

the next few years we're going to have hundreds of millions of these engineers doing stuff. What are they doing? They're creating code, right? Um I don't know if you anyone has looked at this article. I highly recommend checking it out. It's a it's it's art. It's not a science. Um, CVI wrote an article called Gastown and he pretty much engineered a way to have a mayor agent

run fleets and dozens of agents that are able to do software engineering deployments, coding in YOLO mode. Um, and he himself uses very uh strong language to deter people from trying it. But this is an extreme of where we're going, right? And bottom line at the end of the day, the order end solution is Nyx. Nyx comes in and simply put because everyone in this room knows

that it's a lot more complex than that. Unifies the baseline architecture of a few steps of infrastructure. Now, simply put, what happens when you have one step instead of five? What happens when you have one step instead of six? It's just faster. It doesn't matter what's happening on the top. Just means that whatever that solution ends up being, that solution is able to be more reproducible, more

secure, more deterministic, and in general vast. So from my perspective, this is what I tell people. Nyx is the foundation on which we should be building this next layer of the SDLC because we now have an opportunity and I I took this we met up with Jensen two and a half weeks ago and he said you guys are all lucky and I was like you know that's

probably I would I would probably consider that differently but uh he said you guys are all lucky because you are able to be at a time where you can redefine and rebuild flows that were static for decades, right? Everything is now kind of the walls are breaking apart and you can now define how the next new age things should work. And I think we have a huge

opportunity as an ecosystem. We have huge opportunity to establish Nyx as a critical function in the new SDLC. It's not the agentic SDLC. It's just where the software development life cycle is going. And the last thing I'll say on this topic, anybody recognize this this equation? It's a math test. We're all about math. That guy. >> Good. You know what AMDall I'll say it because we're being

recorded. Amn's law is a pretty basic mathematical equation that just says that, you know, if we simplify it, the fastest we can go is the slowest part in a multi-step process. Now, everyone out there is shouting how quickly we can generate code and how amazing that code can be generated, shipped, deployed, produced. Oh my god, that's incredible. But what about the infrastructure? And I think that's where

Nyx comes in and I think that's how we can think about it. I think that's how we can talk about it, right? And we're seeing it. We're seeing it happen because we're not the only people in the room that are moving to establish more on Nick, right? I mean, we're seeing it around this conference. Like this conference is growing. there's more folks coming in from enterprises that

have never heard of Nyx before, but they're seeing that there's a problem and right now Nyx is one of the only solutions that can solve it. So, um, obviously a little bit about Nyx and the numbers, we're growing across the band, which is exciting. It's great, but it also means that we're paying more. So, I don't know if folks were were interested, but we're currently averaging 281

additional gigabytes in our S3 per day. Per day. info team is gonna talk about that and they're gonna have some collely actions and there's things that we can do because at the end of the day again going back to that sustainability we need to think about how this actually continues to grow and flourish and after all the my other friends across the steering committee and the different

teams talk I'll have some quick points on it. Um if folks are wondering how much price has grown we're almost hitting 20K. Good stuff. And one of the last things I'll say on on these parts before I hit the teams is I always think about it this way, right? If anyone's concerned here, if anyone's thinking about their jobs, what software engineering means, right? Kelsey and I were

talking about it yesterday. U he was mentioning that he went through six title changes in across his career. Um things will change and that is okay. What won't at least for me and if it does then then I think I wouldn't be interested to be part of this anymore. I'm saying this very strongly is contributors and community always come first. And I think that's one of the

most important aspects of open source and that that's something that's human to me that I hope that we can continue and we can continue to build up no matter what changes outside and what we can use. Now with that said, I think that we have to work on AI workflows. We have to see how we're utilizing AI. We're utilizing agents to improve the things that we're doing

inside this ecosystem because again, community comes first, but we have to also move forward with the rest of the industry. And my only ask here is share. If you have a flow that's working, if you're using something that's AI assisted, agenticass assisted, uh that is genuinely not garbage. People want to know and and we won't judge you as long as you're upfront and you set expectations clearly.

The one thing that I hate the most and I make this super clear also inside of Flocks in terms of culture and how I operate is if someone sends me something AI generated and doesn't tell me it's AI generated. I'm totally fine with seeing something that's AI generated because I think it's okay. You can write an email. You can even send me a meeting summary. Let me

know it's AI generated and that you've reviewed it. Because the one thing I hate is if I read something and I'm halfway through and I'm like, "Oh is this AI generated?" And I and I just spent 30 human minutes reading something that no human has even checked before without my consent. Right? When I'm talking to an agent, I know I'm effectively doing that with myself. that's a

little bit of my my thoughts on where we are. And like I said, four hours of sleep, newborns, you parents are great. Um, I don't know how you do it. Anyone who's decided to have more than one child, we'll need to talk after. Uh, uh, oh, okay. Um, all right. A little about the teams. What's going on? So, Nixos Foundation 2025, a lot happened. Um, I don't

know if you know all the friendly faces. I am the solo representative uh right now at the conference from the foundation. Uh, I haven't introduced myself at the beginning, but I'm Ron. I'm one of the founders of Flocks and I'm the president of the Nixos Foundation. Um, and and obviously now I I wear an additional hat as an employee of a three-month-old called Dove. Um, yeah, literally.

I mean, he decides when I eat, when whatever. Uh, but we have, you know, we have Lasis, we have Silvin, we have Sebastian, we have Ryan. They're amazing people. They're volunteering. They're not getting paid for this stuff. Um, and they really care, really, really, really care. And for those that um are wondering what's been happening in the foundation, a lot. So, 2025 was kind of the first

year that we fully transitioned into this new foundation model. If folks are not familiar with it, there's a steering committee that is going to come up in a second and talk. The steering committee has a function in the ecosystem. It's voted in. There's a vote that happens every single period. U new folks come in, they're representative of you. They're representative of contributors. They're representative of the people

that vote. Um and then they have a lot of they're very much empowered to make decisions, move things forward, allocate funds. There's a lot of things that happen. And there's also the foundation um that is kind of in charge of keeping the lights on. And I have a little slide of what that means. But we've been really focused on that transition. We've been really focused on extending

partnerships. Um, I think we have we have a good few brewing as well that I would be excited to talk about. Uh, and again, there's a list of of stuff, but you can reach out to anyone. There's no bar. There's no ambiguity or uncertainty. If you're not sure about something that you need, reach out to any of the faces that you see up here today. They'll try

to help you out. They'll try to connect you with the right person. Uh, we like to keep finances very, very transparent. That's what we do. We started this a few years ago. Um, everyone here will have access to this to this to the slides and you'd be able to see um the currently the foundation still does not receive enough funds to be able to do what I

think is is the sustainable I would say P&L. Um, I want the foundation to be in a position that we can fund and continue the infrastructure and keeping the lights on for at least two to three years without relying on anyone. Right? That's what we need to be able to do right now. We're not anywhere close to that. Um, we have partners over at AWS who are

helping us fund the S3. We have partners over at Fastley. We have partners on a lot of different places that are helping us kind of fund these efforts. Um, and and I don't have the full partner list, but there's more partners on the website. You can check them out and and also we're super appreciative of them. Uh, but right now we've been able to keep the funds

somewhat stable with continued funding. Uh but this is a big deal and um and yeah, I was supposed to disclose that this is not yet finalized. The finalized report will come out on the discourse. Um like I said, there's continued partnerships efforts that are happening. So if you're out there, if you're in companies that have ways to contribute doesn't have to be cash, it can be uh

I think um a pier send a machine out to NYX events. Why not, right? Testing it out, showcasing functionality. That's always good. That's always ways to engage with the ecosystem. Um, Nova Custom came in and and and got involved. Uh, who else? We got deep computing, opened up WRT. But bottom line is happy to brainstorm ways to contribute to the ecosystem, whichever way it could be. So,

if you have one, shoot it at us. We'll have to talk to them um and provide some in incentives. Um, one other thing that launched that's really important to be familiar with the NYX sponsorship tiers. So this is a further way to construct and structure out the way that we work with organizations. So today you can go to the website um if you want to become just

a Nyx Foundation and a Nyx ecosystem sponsor there's tiers for each tier you get some stuff uh we try to justru structure it a bit more so organizations can actually come in and get involved with the ecosystem and help fund different efforts. Uh but like I said there's a lot of ways to get Last on this point is that there is a constitution that was written out

that pretty much defined this entire governance structure. I folks that are interested check it out. U it clearly and very articulatively defines kind of foundation and the work between us and how we operate and how we work. I will say that last time I was on this stage uh six not this stage six month ago at Nixxcon I talked about a grant program and we're very close

to shipping it out in draft mode to the community to go look at it. Uh pretty much the entire idea is a more structured way for people with money to find ways to fund the things for the people who want to go work on them. Very simple model. Uh but we just uh ran this with the SC and there was a question that came up with the

constitution because there was a module there that in the constitution the SC needed to be part of and we're like yep let's go make sure that that works out. So we're really taking that constitution seriously. Um and with that said who's the next steering committee uh tribute? Looks like Julian is looking around. Julian slowly making his way up. Sweet. Alrighty. So, we're gonna go in and talk

to our awesome community members for a few minutes, and stage is all yours. >> Hi, guys. >> Hey. >> Uh, is John here? John Ericson. Paging No, he's not here. All right. Uh so uh the two um new people here, it's myself, Phil Taran and Julian Ma. >> Yeah, Julian. >> Uh we successfully campaigned for would you like to have more uh transparency and accountability. Uh we

have done some of that, not all of it. Apologies for the uh the things undone. And uh we're excited about the future of Nyx as always. And um >> maybe we can yeah maybe we can say a word of of what the steering committee is and what's the general idea of this uh governance body. The idea is to uh to to set the direction of uh where

we want to go with the community both like in terms of governance and some technical aspect. And the way uh to do it is to try to um uh to empower different set of people the teams to do their job and to unblock them and to make it as smooth uh as possible for the people that do the actual work to to do their work. To that

end, um you've seen an announcement that uh there's a team that's bootstrapping uh community team uh that is the I suppose new name for moderation and uh we're excited to help them do their work, but they're uh in charge of figuring out uh the terms and conditions, if you will, for like what it means to run the next community spaces. Um hi, there's John Ericson. What have

just made it um you are in the middle of describing the bootstrap moderation team. >> Yeah. So the bootstrap moderation team uh expect some more posts about from us or from them on the matter. Uh but uh that's not out as of right now but soon. uh anything else that you'd like to are we uh go for? >> Yeah. Um we have uh obviously a lot of

like topics and things that we go over but are there any topics that we are not aware of? Are there any conversations that we are not having that we should be having. So if you have either ideas or proposals or things that are ambitious that you don't know if they are being discussed or you think they should be discussed, bring it to us. >> So yeah, please

uh talk to anyone or send it to us or figure it out how to how to get it to us and that will be helpful. Thank you. Uh there's a um actually on the previous slide so there's a a meeting notes in a public git repository. One of the ways to contact us is just open an issue in that repository if you'd like to um you know

uh engage a discussion in public like no one's taken advantage of this yet but it's totally available. Uh and if you uh we don't have a reliable uh cadence of posting on discourse but we do have a fairly reliable cadence of updating the meeting notes on this public GitHub repo uh nixos steering committee. Um uh yeah so the the steering committee exists per the constitution as Ron

was pointing out as to act as an arbiter of decisions that didn't have a central authority before. So there's been a few controversial decisions for instance about like uh the job postings from various companies on discourse on um specific somewhat controversial packages to at least some people in Nyx packages uh whether or not the teams whether or not Nyx packages has a team that's in charge of

Nyx packages itself. The answer is it does Nyx packages core. It's working great. and uh for instance, most recently, where should we have Nixon? And we decided it's in gonna be in Krakow, which is an ancient free city. Um so it's it's going to be great. Um anyone else want to think there's another slide. uh conflict mediation, governance, assisting with fundra fundraising. Um we're basically everyone who's

in the the uh the steering committee also wears other hats and we don't wear our steering committee hat most of the time. It's I'm just a normal contributor on Nick's packages and get the red X just like you do. Uh so uh anything else, guys? Yeah, contact us. Um, we uh actually uh love to be contacted despite uh perhaps external experiences. >> Infra. >> All right. Um,

by the way, there is a steering committee slashfoundation in the singular form aka me panel. It did move so we can uh shake some things around. So, that's going to be at 2:30, not 1:30. Keep that in mind. Um, prepare some hard questions. The steering committee said that they want them. All right. Uh, infra team, welcome, Jonas. I don't know if you guys know this guy. Hey,

everyone. Um, yeah. So, the infra team is probably one of the oldest team in the Nex And the main job of the infra team is to keep Hydra running. Hydra is the CI that is customly built for Nex and that builds all of Nex packages and that produces a lot of derivation outputs that are getting pushed to the S3 cache and um yeah like shout outs uh

especially to Hexa and VQat who are the ones that are kind of on the hydra and keeping things running. And then um we have also sub teams for example the JFly helps a lot with some of the non-critical things like we have a ticketing system um uh email system sorry >> mail server. Yeah, thank you. And um yeah, so in the last uh six months we we

started um so we are replacing the Hydra Qunner. So that's the CI can scale and we also had to mitigate some butt abuse with Anubis and um yeah, it's just doing some like good job and uh The other exciting things that happened is we did our first ever garbage collection of the binary cache in January. >> Yeah, it took a a long time. uh we had to

backfill the whole history of like read back all of the cache and we were able to remove a 100 terabytes of just ISO images that nobody is really using it. >> So yeah, it's a shout out to Brian McGee who did like all of the indexing and research around that. And uh yeah, that's it. If you want to help out, we're we're like available. We always need

more help and uh you can just reach out to the github nexusinfra repo create an issue or come to the matrix channel. All right. Thank you. >> For those of you Tristan, while you come up, not familiar, uh Jonas is one of the OG folks that when we were like, you know what we want to do about I don't know, was it four or five years ago

now? We want to go like bootstrap foundation stuff. Jonas was like, "Yeah, I don't know. I don't know why he agreed to that." Um, so for those of you that are not familiar, Jonas was kind of part of the initial uh, Bootstrap team that put a lot of thought into what ended up becoming it today. So, thank you. Thanks, Jonas. >> Hi, I'm Tristan Ross. I am

one of the members of the hardware team, which I believe we started up around mid last year. Uh so Mick 92 uh Rabbit myself uh started the team. Uh then we got on Zero for a or yeah that name. Uh we got we got Magic then we got Magic RB brought on few months ago and then just yesterday we got uh I believe is Pinbox is their

username. Uh they got added to the team and our focus is hardware. If you kind of can guess hardware is >> oh >> hardware is you need it because you need to run Nixos on something. So that's kind of our job. It is to coordinate between companies and individuals who want to use systems uh and we make sure they work with Nyxos. And so a few of

the things has happened like framework they've been working with us to be able to get Nexos working perfectly on their systems or new enough that it's uh pretty good support. Uh we got deep computing. So they are a risk 5 vendor and we've been working on porting Nixos over to their hardware and hopefully we'll be getting new hardware soon. Uh, and then OpenWRT has also shipped devices.

And we've also been working on Nixo's factor, which allows for better configuration of your systems like, and it makes things a little bit more modular with what devices you have on board on your system. And we've also been working on getting Nixa support for the Fiab Duo. And we're increasing risk five or we're investigating risk five support. And hopefully we'll be getting that around. Uh later today

there is a talk that Yun and I who's the CEO of deep computing will be giving to talk about risk 5. So if anyone's interested that'll be at around 2:00 I believe it >> Yep. So 2:30 I believe. Yeah, that room risk five. >> Cool stuff. All right. Uh John, you want to come up here again for funsies? And JFly, you're right after, so make your >>

Sure. Um yes. One of my other hats, as Philip mentioned, is being on the Knicks team. Um, similar to last time, we continue to grow the team. Love Segvault and uh, Red Vendy are new members. Um, that's why they have the little star icon. And, um, there's, uh, much development work um, remains like new features and stuff, but I think it's especially nice to highlight um, some

of our housekeeping and sort of um, how we work stuff. So, we have release automation recently. That was uh um Sergey, the first guy. I'm not going to try to pronounce that name. Um that nickname and Jük uh MC92 did a release automation. So, now we can have more frequent and easy releases. Sanitizers um just taking advantage of what Clang and GCC already give us um to

detect more bugs. Um, we've done a huge push on perform performance. Um, shortly after last Nixxcon, um, I think our all-time performance improvement increase, uh, Sergey was able to reduce the memory from 30% going into, um, Nyx 232. that's that was that was fantastic. Um, and then a bunch of network uh network relating overhauls too. um loves SegFolt did a huge rewrite of the S3 store making

it no longer use the dubious AWS SDK and use curl more and also just improving how we did things with curl in general. Um so this is those are all things that are completely done and out today um in release next. Um, and then coming up soon, the new installer based on the DETIS Nyx installer that is um in beta thanks to MC92's hard work and that

should be rolled out as a default very shortly. Um, better unified content addressing and dynamic derivations for me that uh continues along actually I may go a little off topic. The next work on there will be in Hydra. Um, but also some new things in Nyx too supporting that. um concurrent and asynchronous eval similar to the pure eval you may have heard about from Sergey. Um more

progress on um lazy paths in particular in our um our journey there to stable flakes which I'm sure you all want. And then finally um we've sort of in in the bits and pieces done some refactors that are getting us closer for Windows support. um and FreeBSD sandboxing. Um FreeBSD sandboxing I hope will be merged later today. And for Windows support, I can at least say that

recently on an unmmerged branch, the uh hello world derivation for Windows was finally built. Okay. Um yeah, very exciting. That's it from me and Nick's team. Thanks so much. >> All right. Um, I'm getting the the get off the stage uh soon uh thing. So, uh JFly, you want to come do like a job? Sweet. I'm Jeremy. Uh you'll see me as Jfly on GitHub. I'm the

leader of the formatting team. We've got a Haskell codebase. It formats all the code in Nyx packages and maybe in your Nyx project as well. Uh if you're interested in HASLL or anything, check it out. Yep. Big shout out to that. We have Nyx packages fully formatted. It's awesome. Yep. Yep. Uh, and then yeah, we're kind of in maintenance mode here. We're fixing bugs, adding features. You

know, we're doing some stuff with experimental stuff like the pipe operators. Uh, we've got one new teammate, Diego. He's kind of the only person on the team who actively writes Haskell. I don't know, so, you know, I'm learning it slowly. Uh, but that's it. Uh, thanks for thanks for using it. You can find me on GitHub. >> Thank you for keeping it short and tight. Um, you

are also with a newborn, so thank you for making it out here. >> All right, marketing team light speeded Dan. Uh, hi, I'm Daniel. I'm uh lead of the marketing team, uh, and also work on primarily branding. Um, unfortunately, the rest of the team couldn't be here. Uh but the rest of them are TLO uh Flo or Florian and uh our latest member I didn't put sparkles

on their face but uh Sigmnificent or Sigma uh has been started helping us out with some new graphic design work which I'm really excited about. Um, but if you're not familiar with our team, uh, we have a few primary roles, which is maintaining the website, uh, organizing the socials, uh, recently developing branding and graphics for the whole ecosystem, um, and also, uh, surveys, which we are in

the middle of processing last year's, um, hopefully coming out in a month or so. Um, so we have some things that we're working on right now that we're really excited about. Uh, Tilho has been working tirelessly to make improvements to the website. Uh, I don't know if it's linked anywhere, but if you go to nixo.org-nix, there's a new page that we've been working on to help bring

people into the Nyx ecosystem, explain it to them a little more cleanly how what Nyx is about. Uh, I don't know if you had trouble when you first came in, but you you hit the Nixon H uh landing page and you're like, "What's Nyx? Tell me about it." And it doesn't really kind of get you there really quickly. It's only through like digging through it a bit,

maybe looking through the docs, looking through the blogs, and you're like, "Oh, okay. I get it now." And we're really hoping to fix that up and also fix a bunch of accessibility issues that have been just lingering around for a while. Um, like I mentioned, we are in the middle of processing the survey. Uh I will tell you right now that we had a 50% increase in

the number of respondents over last year and uh if you've ever seen like the kind of growth curve and like the number of contributors that is definitely real. Uh the whole uh distribution of uh contributors has shifted hard into like the less than one year less than two year. So we're getting a huge influx of people coming in which is really exciting. Um uh let's see ah

for um for socials actually uh I was talking to the team and if you have a meetup uh an event or a cool project that you think people should know about, hit us up, let us know. Uh give us a little copy like a little blurb to say and we'll we'll throw it in our socials. Uh we want people to know about these things. We want people

to like have like a centralized place to like get their news. So if we can start funneling things these things in uh uh hopefully uh the word will spread you know more uh uniformly rather than just being like oh I stumbled upon this or I heard about it a year afterwards through a friend. Um so let's see what else and right Sigma really excited what they've doing.

If any of you got to go to Fostam saw there was a NYX booth, uh Sigma had created this really amazing banner design in like a few days. Um I'm really excited for the the work that they're uh they're spooling up on right now. Look forward to the next release. I'm really hoping, no promises yet, but I'm hoping that we're going to start having desktop wallpapers as

part of the release. you know, like I really I'm really excited about this idea that, you know, you get the new release, you put it on your your uh your your laptop and you go to a coffee shop or your hack event and people are like, "What's that?" You know, I'm really trying to find ways that we can uh bring more attention to ourselves. and I think

the final note is um if any of this interests you, let us know. Uh there's only four of us and we have more than this amount of work to do. There's many other things we would love to do, uh, like spinning up more blog posts, uh, newsletters. Uh, I might need someone to take over doing surveys. so if you're interested in like helping spread the word or

have any kind of like artistic, you know, inclinations, drop us a line in Matrix chat. Um, yeah. >> All right. And I'm gonna call up uh Connor. Uh before that, there's a security team. They're not here today, but Hexa and the crew over there are doing a lot of amazing work. These are one of the areas that as an ecosystem, we definitely need to be investing more

into as more companies, individuals are looking to NYX to become that foundational lower platform for what they're building. Um so definitely check it out, talk to them. If you have some background there or interest, reach out, take part. Um, and with that, Connor, single hand. No, I'm kidding. Connor is a big huge part of making CUDA happen on Nick. So, thank you. >> No, you were right

not to clap, but uh yeah, so I'm I'm on the the Cuda team. There's four members now. Uh I'd like to thank Gayton and Yuri, our newest members, for joining and helping out. uh no one other than me was able to to make it here today. They're all largely abroad. That is to say, not in the United States. Uh but the goal of the CUDA team has

always been singular and that is making Nyx and Nixos the first choice for users of CUDA accelerated software. Uh so that is a very broad very difficult goal. It is challenging. So I I do invite you all to to get involved and I'm going to talk a little bit about how in a moment. Uh but first let's talk about what we've done so far. Uh since we

last presented in October at Nixon, uh we've landed a giant rewrite of all the CUDA packaging inside of Nyx packages. All of the package sets are now constructed almost directly from Nvidia's manifest that they publish. That's going to be huge for increasing the rate at which we can get updates out as well as making maintenance easier. It doesn't solve all of our problems, but it is a

good first step. Uh NixoS 2511 was released. It supports CUDA 12.6 6 through 13 and on master we additionally support 13.1 and that was one of the first CUDA releases that I myself didn't have to author one of our newest members Gayton was responsible for doing that and pushing that across the finish line so thank you Gayton uh in terms of general stuff about the team we

have a new website nxosshycuda.org or thank you to Jonas and Numtide for hosting that for us. Uh we've also got a GitHub or of the same name, Nyxos Cuda. One of the new repositories that we have is one called CUDA Legacy. This maintains all of the different CUDA package sets that have since aged out of Nyx packages. CUDA integrates fairly tightly with GCC and Clang. And so

when old versions of GCC and or Clang are removed from Nyx packages, so too go the Nyx packages CUDA package set. So you can find all of those uh older tool chains and versions of CUDA vendored and supported there. It is on best effort because it is quite tricky to keep that stuff building with the newest things. But if you need to use an older release for

a little bit longer, CUDA legacy should be the first place that you go. Uh in terms of infrastructure, we have our own Hydra now. We are uh building caching and for select packages even running GPU enabled tests and that's fantastic. We didn't have that before and if you've used CUDA you are familiar with runtime failures. The goal is to spot those earlier. Uh coming soon, we're always

working on better documentation. That is a perennial task. If you're using CUDA and you get confused or you don't know what to do, come join us in our Matrix chat room. We're there to help. Let us know where you're stuck or what you're having issues with because that's failings on our part in terms of documentation or tutorials. Uh we've been working on a metadata layer to better

track all of the different constraints and dependencies that Nvidia's packages have both between themselves and with other packages like PyTorch. And uh most recently, thank you Silvin for pushing the problems RFC implementation across the line. We're excited that we'll be able to offer better evaluation errors via the problems implementation. So if you try to build for a known not working configuration instead of obscure error messages telling

you to allow unsupported systems you're going to see why we're trying to stop you in terms of things coming forward on the nixcuda.org page and the repositories associated with that. GCC12 is not looking too good upstream andor has already been removed. So that's going to find its home in CUDA legacy and we're working on our Hydra to allow rolling stable and release channels as well as doing

more GPU testing. So in terms of involvement, if you have a CUDA enabled package that you use or you enjoy using, try writing some tests for it. A lot of the functionality that CUDA enables is all runtime and that's very hard to test. Uh so feel free, jump in. We're working on documentation that says here's how to get started writing this test. But we sure could use

the help. >> And again, huge hands to Connor here cuz I do not know where we would be without this team. I will say really quickly. So, we're going to we're going to start wrapping up real quick here and we're going to have some uh one give give us two more minutes. Uh there's actually a few things that I hope we'll be able to announce in the

coming weeks around CUDA, around uh partnerships in that domain. And honestly, the only thing I want to say is that if you're a part of a firm or care about CUDA and about this ecosystem, reach out to us. Reach out to CUDA, reach out to me. I'll get you involved whether it's as a company that wants to fund these efforts or as an individual that wants to

take part in them. Really, really important stuff. Um, this goes into the what we can do category. There's a lot we can do. We've been doing a lot as an ecosystem. You have been doing a lot. The contributors have been doing a lot. Keep that up. And I think we're going to have another amazing 2026. And I'm very, very excited to meet you guys over at NyxCon.

And the last throwback is this is what the first NIXCON looked like right after CO. It was about 140 people in a windowless Parisian basement. So, we are only getting better from here. So, thank you guys. Thank you for being here. And um we'll we'll stop with this one and I'll introduce the next speaker in a second. Sweet. So, one of our partners, I want to welcome

Anishh from Enthropic over to the stage. Um, he's going to be launching the new version of uh cloud right here. >> That's what that's what I heard. >> Is that Nixxcon 27? Is that when we become important? >> I don't know. I don't know. >> Hey everyone. Uh, I'm Anishh. I work at Enthropic. Uh, first of all, thank you for the organizers, the community for making Planet

Nix happen. This is our third year here and we're really glad to be sponsoring series this year. So, thank you all very much. If you don't know us, Enthropic is a AI safety research lab and we make claude. Uh, so if you've heard of Claude code, that's us and that really means two things being an AI. One is we're moving at hypers speed. Everything is moving at

the speed of light and the other thing is we're always at the frontier. We're always doing things that haven't been done before and you know, occasionally breaking our systems and our vendor systems along the way. And that means a lot of imagining from first principles because it's things that haven't been done before. There's no playbook. Uh and so that's one of the big reasons that we've making

a big bet on Nyx to kind of build and deploy all these systems so we can have all that control uh that precision and that reproducibility. Um we're also been really uh pleased to be able to be contributing upstream not just little bug fixes but larger features as well. So we just landed a rewrite of the S3 substitutor in upstream ny. It's way faster and it's also

a lot more reliable. Um, and we got enough contributions that Bernardo actually joined the NYX core team this year. You might have seen on the slide. So, we're really happy about that. In addition to upstream, we're also working with some of the other, you know, great organizations in the Nick space on other improvements. So, things like faster evals or better and more ergonomic packaging. Sam gave two

killer talks yesterday. Bernard talked about Nyx for remote builds and what's missing and how we add that. And Sam just gave the most the funniest deep dive story about how we like our whole development environment just to make Nick's builds work with sandboxing. If you didn't catch them yesterday, definitely watch the recordings. They were absolutely killer talks. Um, and I will say if you found those interesting

or if you just want to come talk to us, we've got a whole nine of us here uh at the conference this year. So come by the booth. Uh we'll be uh around. We have a lot of weird problems with Nick. We'd love for you to come help us figure out how to get out of these weed corners that we've uh found ourselves in. So, uh again,

thanks everybody. Really glad to hear uh be here and here's to a great planet Nick. >> Thank you, Anish. All right, last one minute from Jonas. Thank you. And then and then I'm fine. We're we're off the stage. >> Yeah, we'll keep it short. Uh hi, I'm Jonas. I'm the founder of Numides. We are a consulting company. We like to build things. Uh we're kind of old

senior engineers. So if you have a startup or a bigger company, we can come in and help you maybe offload some of the things you don't want to focus on. Um yeah, and when we're not working, we like to build things. So we contribute a lot to the Nyx ecosystem. If you use the flake utils or system manager or things like that, that's us. And yeah, we

like love to build things. So excited to do more of that in the future. And last thing is we are running a hackathon. So if you want to hack on something with Nyx, build something around self-hosting, we you have the whole day and at 4 p.m. at the on conference, you come. We do demos and there's prizes. Uh we have RAM as a prize. All right, that's

it. Thank you, Jonas. All right, I'm getting off the stage. Thank you guys. Have a great second day and we'll see you all >> All right, I know we ran into the break a little bit, so we will start the next talk at 11:05 so that you have time to go to the bathroom, get water, do what you need, and we will see you right back here.

11:05. Thanks, friends. All right. All right. All right. Hello friends. If you can make your way to your seats, get settled in. We will be starting the next session in less than one minute. All right, thank you and welcome back to day two. It's been a fun and eventful morning and we are excited to continue going with the sessions. So, if you could help me to give

a warm welcome to Craig, he will be giving your next session. Hello. Hello. Testing. Loud. Quiet. So, hello. My name is Craig Jackson and thanks for joining me. I've been watching Nyx OS as an observer for a while now. And over the last couple of years, there there has been a lot of movement in the CI/CD and build spaces of Nyx OS, Nyx anywhere, dev environments, even

microVMs. And then AI happened. And with AI, I really think that all of the inherent barriers that kept Nyx OS from being used and was a nervous choice in the past, I think it's finally eroded away. And it the time is now to start using Nyx. And it's gaining ground out on the network edge where I work in monitoring, alerting, buildr runners, and infrastructure testing scripts. But

first, the obligatory who am I? Well, my name is Craig and I'm a redneck adventurer when I'm not working climbing mountains in these off-road machines. I'm a synthesizer enthusiast. I like turn buttons and push or turn knobs and push buttons. But process and automation has always been a part of my life. I studied horiculture in college where you automate the life of a plant. I worked in

health care as an orthopedic surgical consultant where I worked with surgeons and hospital systems to increase efficiency and minimize waste. COVID let me leave that industry and I'm happy to return home to the internet because I found IRC in my teens and I've never left and the internet is home. So I finally came back to it and now I'm a DevOps engineer at a company called Net

Actuate and we are a global infrastructure cloud provider with a network first focus. So we have a managedcast network that our customers use for low latency connectivity to their end user. We even hold the Hroot DNS server on our network as the low latency connectivity and anycast is kind of our bread and butter. And so what we're going to talk about today is the importance of the

edge network, how significant it can be in both cost and management requirements, the difficulties in setting up that infrastructure, and then we're going to explore how Nixos adoption is being increased out there on the edge in these singleuse cattle type machines. I was going to end it with trusting in Nixos, but I'm at a Nyxos conference and the the talks yesterday and today are more about how

to give the messaging to other people and recruit other people to our cause. And so I'm going to modify the trust at the end a little bit because I believe that Ron and Kelsey had the message correct yesterday. But to get back to the network edge, I'm going to use a few terms. One of which being POP. I'm going to say that a lot. It's just a

point of presence where we announce your service across the globe to bring your enduser into it. But the network edge are multiple pops kind of loosely fabricked together to create multiple entry points into your service around the globe. It's important to maintain an edge network and to build it for multiple reasons. The closer you can keep the assets to the customer, the less bandwidth charges you'll incur

over time. As you increase regions and your service grows, you can rapidly scale with a new pop. And it can also give you data sovereignty compliance. That's becoming very important as the geopolitical system falls apart. And so more and more countries and more and more regions want you to keep their their uhi civilians and their citizens data in the country that they live in. And ultimately when

you do have a resilient and reliable edge network, your customers and you enjoy low latency connectivity. And so what type of infrastructure and what type of workloads makes up an edge network? Well, again, we're bringing people into your network and into your service. And these machines are typically singleuse machines and so they're quite perfect for Nyx in my opinion. Such roles would be like a BGP inress

Damon or a reverse proxy into a Kubernetes cluster, firewalls, load balancers, etc. depending on the service that you're providing and the needs of that service at the edge. There's a whole lot of different ways to instill this infrastructure on the edge. We have all the infrastructure as code tools. we need to manage multiple data centers. So there's infrastructure management tools. There's networking appliances that we need to

connect together in all of those different locations. And all of that's loosely tied together with git runners and web hooks and everything else. And once you have that infrastructure, we need to alert, we need to monitor, and we need to observe. So there's a whole lot of infrastructure that surrounds your service on the edge. All of it's important and all of it can be dangerous to maintain

over time. And so these challenges present itself in multiple ways. And the Nick's answer that we already know is really the configuration drift. We need to be able to trust what we're doing and we need to be able to rely on it consistently over time. And on the edge, since you're not in control of the data centers, the locations, and you're not control of the hardware, and

who's going to be working on it, it's just timely to maintain. It's expensive to maintain. And when these systems start to fail, you can no longer trust the infrastructure on the edge of your network. And all of this leads to increased costs over time. not only in managing the infrastructure itself, but the configuration spread out over two or three different systems. You've got Terraform and Ansible. And

when you're using multiple points of presence, you really would benefit from having Netbox and AWX. So now we've just increased our amount of infrastructure that we need. There's also time spent managing all of those. And since it is scattered and we have all of these little pieces of data that's all over the place, who's forgotten to update the CI/CD bars and pushed run? Who didn't notice the

GitHub runner didn't finish yesterday? We run into those problems. And the Eureka moment with Nyx came for me when we enabled the ability on our platform for customers to install Ubuntu. They can go in and customize it. They can edit their SSH keys. They can add their ingress rules. They can give the basic bootstrapping already set up in the machine and then they can lock that machine

down and clone it and deploy it in multiple regions already bootstrapped and they've loved it. They instantly started creating multiple packages, one for each region. And now we have so many private images, one for each region, one for each service. It's rapidly getting out of hand because we have 45 pops and if you maintain an image in every single pop, well now we're looking at 207 gigabytes

of storage just for your bootstrapping images. And so these manage managing the infrastructure on the edge gets costly and it gets very hard to keep the configuration in a constant state. And so Nyx was brought to me by a customer and she was trying to figure out if she could run services that were single systems with lengthy policybased configurations. So caching systems, BGP ingress, firewall rules, you

have to put them in order and they all relate to each other. And if you mess up that rule set, the system's broken and it doesn't work anymore. And so it's easy to roll back. It's quick to rebuild. And she wanted Nyx out on the edge. So I confidently and arrogantly said, "It's just Linux. Of course, I can get it to work on our system." And she

said, "Great." And she sent me this. Well, I don't think KVM can deploy a Flake. I don't I didn't know what to do with it. And this is why I didn't use it. This is why I didn't use it for years because I can do it quicker with with Ubuntu at this point. I have to learn this first. And so this is where I'm gonna pivot a

little bit and talk more about what Kelsey and Ron have been talking about. I trust in Nyx now because I understand this enough to get it started. And I'm also not a tinkerer. I'm not one of you guys. I use it strategically like a scalpel to fix one singular little problem. I use that compute outside of my Kubernetes cluster. And so I don't think that trust sells

to other people well because they trust They trust this. And when I tell them that I don't trust this, they say skill issue. I trust that my Terraform works. Why do I need to learn a whole new system? Well, because this, as to Kelsey's argument yesterday, isn't consistent. It It's nonconverging. I have to go to 17 different uh 17 different systems to understand this. But what works

for me is I can now begin to converge my configuration down to a base flake. And the next customer that showed me how to use it effectively in a team opened up my world and now we're using Nyx internally. I'm no longer just watching customers use Nyx. I gave this talk internally last week to our company so that I can practice and get their thoughts. the tech

lead. So my role in the company is I get to prototype with our architect and then when he's done building what I helped design, I then brute force what he built and try to promote that to the customers. Here's what we just made and here's the cool you get to do with it. After I gave this talk for the first time, he said, "Do you think I

can get our Kubernetes worker nodes for our managed Kubernetes service on Nyx?" I said, "Do you think I can?" Yeah. Oh, do you think? And he said, "Well, I really start need I need consistent boot times now. We need a predictable deployment strategy and we need a quick deployment for Kubernetes workers nodes. It's currently slow. And so now that it's consistent and it's quick to boot and

it's faster than Ubuntu once it's programmed, he's in. I've got his buy in. Not because I trust it. He doesn't trust me. It's consistent. And it's when you have everything built in. And by the way, the only reason that he's going to use this is because I've built the configuration. He just needs to load his service into it. I built the flake. So I've given him the

devshell that he needs to interact with the service that he's going to put on my in network.nix. This is ephemeral. We inject cloud init secrets into the machine, but we're not going to rewrite our ISO vendor data attachment because it works for every single of our other operating systems. And so Nyx doesn't persist the network. That's the one caveat, but that's fine because you're going to rebuild

it with your own system anyway. You can persist the network and we are we automate that with a cloud init script. So inside cloud innit, you customize your base flake and then you can download the ro that that flake needs to inherit. We've now reduced entirely how many images we need. We need one. and we can download the ephemeral role and reboot it into that role anytime

we need to. And so now we have a seamless build process and we've got that comfort and trust. We can treat this like cattle or pets because it's truly reproducible. It's truly declarative and it's truly software. And so if we look back at this role listing again, the first customer brought it to me with BGP and proxies, but that configuration is a pet in my opinion because

we're going to continue to iterate on it. We're going to roll back and we're going to nurture those. The top five or six all kind of feel like pets. The configuration is very similar. And so if it works for BGP, if it works for proxies, it's also going to work for all of these others. And so again, the cattle I think we can replace GitHub runners. That's

how I'm using it internally. Now I am getting rid of GitHub runners entirely. They're going to charge me for my own time that I'm going to run my own runner. No. Also, they're going to give me shitty implementations of their runners for me to interact with. They lock. They freeze. I have to go in tomorrow and make sure they finished. I can do better with Nyx OS.

And I can do it better with just web hooks out of GitHub because they let you attach a secret to it. So now I have a listener of web hooks and the secret is the role that I want to boot and I just have a whole bunch. I'm an infrastructure cloud guy and I hit every hammer or every nail with a VM. So I have a whole

bunch of Nixos machines turned off just waiting. And I have a listener that receives a secret and turns on that roll, it does a cron job and it turns itself off. And so I have a fleet of single use, quick action, quick to boot, persistent, consistent And it's just a fleet of runners. With that, I can use Kubernetes jobs and still have Kubernetes orchestrate my little GitHub

runners. And so if it fails, Kubernetes fails, it runs it again. And so I can interact with my little running system and create a whole bunch of builds. And I can start to add in just little singleuse pieces all scattered throughout the infrastructure around my team. And they have buyin because I've made it easy for them to And so in summary, the edge network is significant and

it's something that you should babysit and it's something that if you manage correctly, your customers will enjoy and you will enjoy as well. It is difficult to manage and as your service grows, the complexity and the size of your edge network is hopefully is going to grow as well. There were inherent problems not to use nyx and that's what we're overcoming today, but those are gone. AI

happened and two things happened with that. All of our bosses are now uncomfort or that are comfortable with approving uncomfortable things. That's just the reality of it now. They're approving AI slop. And guess what can be part of AI slop? Use nyx. They don't know what's in the code. They don't know the infrastructure it lives on either. and they're rubber stamping it. Let's use Nyx because in

this I'm using it because I trust it. I came from out of industry. I understand process and I understand automation. However, I missed containers. I came into the cloud after Docker. Kubernetes is an amazing machine to me that I finally understand enough to interact with it and I love it. But there's inherent problems with all of this. I love running workloads in Kubernetes. I hate finding the

workload that failed. And so in all of these abstract systems, I've lost that intimate touch with the data. I've lost the intimate touch with the infrastructure. I've lost the intimate touch of all of it. And I'm just observing it now. And the only thing that I can trust in all of this abstraction is a configuration or a flake.Nix Nyx file because it's all explicitly listed. And when

you have it peppered around all of these abstract systems, it's like bumper cars in bowling. You pull out the little bumper and you keep this abstract system going down the middle of the lane. But if you don't have these rigid constraints built there's no comfort there. And so what's crazy to me is when I first looked at Nyx, I saw chaos and now I see order because

the rest of the systems around it are getting chaotic and this is comforting. This is calm. And so in closing again, my name is Craig Jackson. I work with a company called Net Actuate. We do really get excited about infrastructure and cool fun projects. So, if you have a Nyx OS project, I can give you a base flake, come and talk to me, the flocks, and the

planet Nyx crew. They've been incredibly helpful. So, if you have any questions about where to use Nyx, where to put it in in your infrastructure, how to convince your boss, how to get buyin on your team, please come find me or them, and we'd love to talk to you and help. Also watch our GitHub because I'm using Nyx now and so I'm going to start creating a

bunch of flakes and workflows that just make it easier to consume. There's not a good example of attaching vendor data in cloud innit on a no cloud ISO and have that data persist in inside of a Nyx flake. And so we're going to start creating examples as well because we really enjoy this community and we're glad to be a part of it. Thank you so >> Awesome.

Thank you so much. We really enjoyed your session. And I like that you called out other people, too. Like I Something that I've really enjoyed about the community so far here is that everyone has been so warm and welcoming and it says a lot about who you are as people, as builders, and just as conference attendees. And we really appreciate that. Um, thank you so much again

for speaking with us. >> Wonderful. and I think we're good to welcome up Yasik. If you're ready, we'll let you get set up. >> Sure, we can do you can get set up. We'll let Yasik set up. If you have any questions for Craig, feel free to ask him. And if you need to go to the bathroom or anything, we've got five minutes. So, we'll see you

back in five. Okay. Screen >> sounds good. >> Oh, yeah. Here. Very nice. >> Oh, okay. Like this. >> Okay. Now I need a second. There we go. Hello friends. We are ready to get back on track here on track one. So again, thank you so much for joining us today. And who's ready to get started? >> Yeah, love it. All right, so I will be handing

off to Yasik. Thank you so much and enjoy. >> Thank you. Thank you for a nice introduction. Yeah. So hi, I'm Yatsk and I'm here to talk with you about the test driver. Actually, we do that today like twice. This is the short talk. My goal here is to um tell you things about the test driver that you might not have heard about it. It's a really

powerful tool and also give some oversight and what's new there, what's the architecture, where to look if you would like to study it because you feel intrigued and stuff like that. No demos. But today uh at 3:30 we do the workshop and there that will be all about demos. So this is like very theoretical and later very practical. If you find anything here interesting, please also join

the session later. Okay, one word about me. I'm the founder of the Nyx Academy. Uh operates under the applicative systems group label here in the US and in the European Union. Uh we've been training more than 600 people in the last two three years in Nyx and Nixos. So really help organizations ramp up on Nyx very often. I know from my own experience that takes years. If

you want to take it months and have everyone the feeling like to gain like why Nyx is important for them too, not only for the orc, we can fix that. There's a price tag to make it in months. And we have several organizations where we trained all the engineers and they really ramped up on speed. So there's a huge ROI that you get there. So if you

find ramping up your ORC difficult with Nyx, contact us. will can help you fix that. Like with all the other orcs, um there's also the Nyx antiatterns book. It's pretty new. This is the last chance to get it for free outside on the table. Please pick your copy. Uh we use that as a little test balloon. Got surprising like overwhelmingly great feedback. So I decided to sell

it now. Um but there's I think there's like 30 copies still left over. Get yours. Okay. um about the test driver. So the Nixos test driver who has already heard about it so far please. Okay, who has used it so far? Okay, a few. So um the test driver is a Python app essentially. And the problem that it tries to solve is to provide you a very

simple way to say okay let's spin up five VMs with certain configuration. One might be an email server, one is a web server and then you do certain things where you simulate a client to do whatever you like. Uh the Nexus project also use it to test their own installer to test uh Kubernetes to test MySQL sharding whatnot. Um everything that can be tested by providing little

environment with multiple VMs that try to simulate some kind of real life scenario can be modeled with this test driver. So if you need complex test scenarios that run quick, this is for you. In general, we basically have the driver, you are operating from the driver perspective and then there's multiple Python objects like machine one, machine two that you can access and start stop. You can power

cycle the machines. You can communicate with them via serial. You can lock in via SSH when you want to debug. So there's a different modes. One is the quick sandbox runs. One is the interactive mode. I will show it to you. Um, you can even run graphical VMs and test graphical applications. You can make screenshots and stuff like that. So, the test driver gives you access to

all these things. And you can also run machines in multiple networks. So, there's blue and red networks on this picture, which kind of shows that you can have multiple VMs running in different networks to, for example, test networking, routing, and whatnot. this is my favorite example of an integration test. It tests a bit torrent service. So the bit torrent service is basically peer-to-peer file exchange, right? And

to test bit torrent, what they did in the Nixos project because transmission is the torrent client that's packaged in Nyx packages and there's also a module that you can activate. They set up a tracker where they have a transmission client that kind of sees a file and then they have Apache serving the torrent link and open tracker coordinating all the file shares and then they set up

a client that is in this red network. So that simulates a home network subnet. The blue network simulates like the internet. And this client is behind the router. And then they would kind of transfer the file. And then they switch off the transmission client and the tracker and switch on the transmission client and client number two. So that the file needs to move from here to there

now which is behind the firewall. So they also test how transmission unlocks the port here via mini UPND and so on. You can imagine that we're not only testing bit torrent here. We're testing Nixo's network configuration. We're testing Apache. We're testing Open Tracker. We're testing net routing settings. We're testing UPND and stuff like that. And that is a pretty complex scenario here actually. And you can imagine

that Nixos can actually get reach this high packages per maintainer ratio and keep them up to date and running because there's a so many tests that track this stuff and it gets very apparent when something breaks but with a little update. However, the infrastructure that we define here is defined in roughly 50 lines of code. You just need to write 50 lines of code asterisk. I'm removing

all the uh bracket line noise kind of but the actual code is like 50 lines that describes all this infrastructure. It's just a few Nixos configurations showed together and that just works. There's another 60 lines of Python that does the test script like create the file, share it, blah blah blah, and the whole test runs in below two minutes on your laptop. Now, so this is kind

of the the simplicity and power of this framework at the same time. Um, there's also graphical tests that you can do. Um, my favorite one is the Google Chrome test in the upper left. So Google Chrome is one of the first browsers that puts tabs into each processes and uh isolates them from each other and they are calling that sandboxing and the thing is to test if

your Chrome is properly configured at buildtime and running with sandboxing enabled. That means that you have to run Chrome graphically. There's no command line flag that gives you that. And you go to Chrome/andbox and there the page says you are adequately sandboxed. What this test does is it runs a graphical VM, runs Chrome, clicks into the middle of the window, runs copy, pastes into a buffer, and

then grabs out of this buffer if you are adequately sandbox, yes or no. And the test is parameterized on different binaries of Chrome. It's really small and it runs also below two minutes. I think they also test multiplayer games. Uh for the client server connectivity, they also test Gnome and there's kind of um text recognition too. you can make screenshots of a running VM and you can

also uh check if there's certain text graphically on that thing which is really amazing. Uh sometimes I put QR codes on these slides so we have on the nixacademy.com blog examples of this that go more into detail. What is also really interesting is that sometimes people say like okay yeah vagrant also enables me to build multiple VMs but I wouldn't build a test with that because that

means 5 VMs means 5 BM images that takes ages to build and then imagine the build infrastructure if 100 developers do that all the time you have like thousands of images clogging the disk what is apparently interesting and I'll show you that in detail later in the workshop session we're not building any images with the bit torrent test that you've seen earlier we would build the so-called

top level derivations. So that is a set of store paths that get thrown into your host nick store while building that stuff. When we launch those kuimo VM sessions or processes, we just map the host nick store read only into the VM and it selectively boots those uh paths to get its configuration there. So you change the configuration from a bit torrent test here and there, change

some host name, whatever, and rebuild and you're running in seconds again. There's no images. So this is really quick, really fast. It's really different than other u tool sets. This is how a test looks like. I have a minil one here. So when I said earlier that bit torrent just needs 50 lines of code, I was referring to the blue part here. That's the list of nodes.

What's interesting here is we're saying machine one exists and machine 2 exists and you're abbreviated. You just add a lot of config that can be random next config. And then what the test driver does later in the test script which is the imperative part. So that is declaratively describing your existing infrastructure without running yet. And then when it runs you can start the machine, stop the machine,

do whatever you like. Here because we named them machine one and machine two, we get a machine one and machine two Python variable. And here this is real Python. I exaggerated it a bit with a for loop but to demonstrate that we're using real Python here. can also import your own packages and so on and so on. And um not only get Python variable names here, you

also get DNS names. So one machine can ping each other using those names as you describe them up here. So there's a lot of default settings that are taken off your shoulders to quickly get something running like this. Um as you can see this is just an attribute set with an attribute name nodes and test script that isn't in derivation yet. You typically run it through packages.runixos

run nixos test and that creates the derivation that puts the Python app with your VM configurations and everything together and then can run them in the sandbox that then produces your test fail or not fail result. So that typically happens in one big step and that's as simple as that. the tests then run in a sandbox right. So the Nyx builder typically builds whatever you like PDF

files, package compilations, repackages, Python stuff or whatever or it just runs 20 VMs in this sandbox. It doesn't care which is pretty cool because this way tests don't only become very reproducible. It's also that your build infrastructure becomes your test infrastructure. So in many companies build and test infrastructure is something different and separate from each other and test infrastructure is really complex. But when you use nyx

and accordingly design your infrastructure, you just have a huge set of builders that built and test and do that very quickly. What is also cool is because the builders just run the same thing in the sandbox as you do. If your CI goes red because some complex test failed, you would just reproduce it on your laptop and then fix it and the first commit that you push

already contains the fix that made it work. So there's no more the CI is the single source of truth and I'm committing and pushing and hoping and oh, it still didn't fix it. So commit and push again and again and again again or press the replay button. You're liberated from that. Everything that runs on your laptop will also run on the CI. Um this is the interactive

mode. So if something fails, you can check out the test and let's imagine you have a test that runs for one hour and then you fix something. Although it runs locally, you don't want a feedback loop that goes for one hour or two, whatever. The interactive mode allows you to run a test the same way as it will run in a sandbox outside the sandbox and then

the VM windows pop up in front of you and you can lock in into them individually. You can sh into them. You can play around. So if something fails, you can look at the logs. You can install more debugging tools. You can really dig deep what the test uh problem was, fix it, and then run it in the sandbox again. Finally commit and push it and it's

gone. It's done. So this is pretty cool because the interactive mode also has this nice text completion. So you can find out what is there, how did the pro the procedures work and then find out how to even test your software. So while you are playing around, you can take your notes and actually set everything in stone for the actual test later. So this is really useful

and really easy to Um, integration tests also run on Mac OS because Mac OS can of course run KU emo natively and then run your VMs inside there. So that is supported. N packages itself contains more than thousand of these tests. It's like last time I've seen with a counting the lines in the Nixos/ tests folder was,100. So there's nothing that hasn't been ever tested. So if

you kind of are like how would I do that? You could look into this folder and check out other tests and get inspired from that. There's wild scenarios that are elegantly implemented that you can uh kind of use for yourself. The architecture looks like this. Sorry, I think the text is a bit too small. So in general, you have like two profiles, the KUMOVM profiles which also

work without the test driver. So you can quickly import the KUVM profile in Nixos config and then spin up a VM in front of you and play with that. The test driver just makes it so that you can orchestrate multiple of those and put them into a network which is of course a little bit more complex. What's new is there's also a systemd spawn container profile that

also works alone and we can now use both. I will talk more about containers in one minute. The test driver is just put on top of that. Test driver is basically a Python script that's bundled with some configuration that can run five or 15 VMs and wire them together, right? And the configuration of the test driver is also not trivial. So you have the test module system.

What's interesting if you want to learn about that in the Nixos modules folder under virtualization you see those profiles. These are worth studying because they are also worth without the test driver. The test driver Python project is on a Nixos lip set test driver. And then the packages testers run Nixos testing with the attribute set that automatically wires everything together. that is under Nixos/lip/Testing which is also

documented because it uses the normal NixoS module system like the Nixos modules too and that together wires up the whole powerful test driver framework. Um there's also new debug hooks. So that is a feature that's really worthwhile mentioning because sometimes people told me like look we have tests that run like for five hours and then they fail but they never fail on the laptop. So what to

do? Um you can enable the debug hook and configure your uh Nyx builder accordingly. Um and then when it fails like next week or something then you can lock into the builder and the test is being frozen for you and you can actually lock into the machines. There are some complicated lines here. The test driver will print them for you. It's just to be mentioned here for

you that you can actually freeze failing tests on remote infrastructure if they are hard to grasp why they fail. then you can lock in and debug the hell out of it and finally fix it. So that's relatively new container tests. So customer asked me like uh last year how would you do GPU testing in the sandbox and we discussed it a bit more and then we realized

look so if you could kind of uh add containers to the test framework that would be more than you're actually asking for but it would make everything also interesting for the public. So the public would gladly accept that into the upstream next packages and then you get what you need and you don't have to maintain it because the community thankfully accepted the whole thing kind of and

everyone profits from that. Shall we try that? They were like yeah let's do that and now it happened. So we are nearly merged. We can now run containers. What that means is that instead of running cool VMs, we just run systemd and spawn containers which are faster because you don't have the overhead of a new kernel and hardware models and stuff like that. Um, and cool thing

is instead of writing nodes name equals blah blah blah, you say containers domachine name equals blah blah blah and that's it. You can basically switch over many tests if they're not graphical and so on. What's also nice is that you can combine VMs and containers and you these are kind of fast and light right so you can write you cannot run 150 VMs on your laptop but

you can always run 150 uh containers it's possible also we are kind of relieving people that used the test driver before to use bare metal infrastructure because you need KVM acceleration and typically nested VMs don't work well um so you needed bare metal infrastructure to run Nixos tests quickly Now you can use cheap VMs and they just run without overhead with containers. So that's a huge liberation

from certain hardware requirements that we don't need any longer. Um and the final goal for our customer was that we can now map GPU devices from SLDEV blah blah blah into the container into the VMs and run CUDA in the V in the container. For example, um this is kind of the CUDA example. Sorry, this is I didn't know that the board would be so small. We

have containers do my container equals Nixers configuration. And here we install sax pi. That is a minimal CUDA application to test if CUDA is configured at all. And here's a list of paths that we make available from the host into the sandbox and then into the container. So we just need to list them up. And then in the test we basically run sex pie. And that program

would crash if CUDA wasn't available but here it doesn't and that's it. So we can run CUDA stuff inside the NexoS integration test which is pretty cool because many real life products are deeply embedded in some Nixos configuration and this way you can finally have very production near scenarios on your laptop basically or whatever hardware has Nvidia cards. Um to enable this we need to set up

the Nyx demon in the config. So we need to allow for autoallocate UIDs and croup stuff. The thing is that um systemd needs to manage a lot of like mappings between user ids, identities and stuff like that. It's really complex. You can also not run any SUID stuff inside that. But even Leonard Puttering, the main systemd maintainer started hating SUID. So that will most likely vanish anyway.

We're a little bit at the edge kind of, but this will improve anyway. And it works pretty pretty solid and well. Um, upstream container support will be available in 2605 in a few months, but I expect the pull request to emerged in a few weeks already. There's a lot of uh review that happened and improved it very nicely and have an example test as you see later.

Two containers pinging each other and then dying again is 2.4 seconds in total with old boot stuff and so on. A special thanks to the clan project who wrote the first proof of concept of container tests um and also JFly and K mine who worked for us to implement this in an upstreamable way which was a lot of work but they did that so perfectly in Mar

27 he sits in Stoutgart works for SEUNet he helped us review that stuff and he also exposed some more optimization potential made which made this amazing if you want to try this out now check out the Nyx packages fork of applicative systems. It's in the branch Nixx test containers. We'll do that later too if you want to see it. Uh one word about the NixoS documentation. People

typically find out that when you Google stuff about Nyx packages or Nixos, you find nothing just blog articles of people and so on, but not the primary documentation. That is because the Nyx packages docs and Nixos docs are one huge HTML document so large that Google does not index it. So until that is fixed, please bookmark these pages and use Ctrl+F inside and you will immediately find

what you need because everything I just talked about is documented already. It's just that you can't Google it. That's the only thing. Last slide. Thank you for listening. So we do Nix training and consulting. I loved this project because we had a um customer sponsor this nice open source stuff and everyone profits from this now and uh our lever was that we of course know the right

people so who to nudge to get complex changes upstream and to redesign this in a way that's actually upstreamable that's we really much enjoyed that um yeah so there's also our newsletter so you can scan this QR code or look for an academy newsletter we typically send monthly updates on what happened in the blogger sphere with nice summaries for people who are thin on time. Uh thank

you for your attention. Do we have time for questions? >> Okay, >> questions, please. >> Hey, really cool talk. Um, I didn't realize all that was there and I just recently reimplemented it all. So that's disappointing. >> I'm glad you like it. >> Although I did do uh mini cube as well. So it does like Docker and then it does mini cube tests which could be we

should talk. Um but my question is that's great to see all the CUDA automation but and and you've also got graphics >> in right so maybe we could actually test the the Nvidia display drivers working inside Nixos. I mean to test GPU stuff inside Kuo you would have to map the PCI device into the VM which is of course possible. Uh no we talked about doing that

with containers which relieves you from actually exclusively taking the whole PCI device can just map the SLDE/ Nvidia blah blah blah paths that are available on the host and share them with other applications running on the host which makes it much easier. You could of course do what you just said, but you would have to rip the whole PCI device logically from the host system and into

the VM. Um, for our customer that was not feasible, but it's generally doable. Yeah, >> I think we have time for two more questions if anyone else wants to ask. >> Yeah, I'm coming. Yeah. So earlier you had said that um you can run Nixos tests on Mac OS and I saw this uh on Discourse when it came out and you were involved with getting this merged.

Um but to my knowledge you still need to have some way of building >> the VM. Um and previously that meant like installing Nyx Darwin having the Linux builder and then doing this. Is there another way? >> Uh what you just described is the easiest way. So the question is can I just run out of the box Mac OS Nixos integration test? Yes or no? No. You

need to prepare your Mac OS to do that because Mac OS can run these tests but not build your VMs natively. For that there's a Linux builder. It's really easy to set up with Nick Darwin. It's a five minute process. We described that in this blog article here if you want to have a look. So, it's generally doable. You just need a quick setup first. >> Last

question. >> Yeah, last question. >> All right, I think we're all good. If there's any is there anything that you wanted to share that you didn't get a chance to say? >> Um, thank you for giving me such a great time here. I really love being in the states. Uh, Planet Nyx is an amazing conference, especially when it compared to Nyx Con. There's some nice tradeoffs and

I really enjoy being here every >> Thank you so much. >> I have a speaker gift to give to. >> All right. Thank you so much for joining us everyone. We are going to cut to a lunch break. you have 90 minutes to get lunch and then we will have sessions in track one and track two. So thanks again. See you at 1:30 and also Yasik does

have a workshop later today. And >> can I say one other thing? So we have we not only distribute the book but also we have these Nick stickers. And what's not apparent when you look at them the QR code is individual per sticker. So that means you can scan it and then claim the sticker with your GitHub account. And when someone else scans it, they see your

GitHub account and your achievements in Nyx community because we track all the pull requests and issues of the public data. And if you ever contributed, it will be visible on that sticker. Thank you. >> Was it the one that you guys ended up? >> Thanks again so much and we will see you back here at 1:30. Bye friends. Hello friends. We will be getting started soon in

both room one and room two. So if you can hear me outside, please start making your way in. We are excited and getting ready to begin. Hope you had a great lunch and thanks so back. I hope that you all had a wonderful lunch and you had some time to rest, recharge, eat some food, and get ready to learn some more. Yay. Nay, how we feeling? So,

thank you again for joining us. Um, thank you to our sponsors who without them, we could not have made this possible. Thank you, of course, to the audience because again, if you weren't here, we couldn't do this. So, thank you so much for joining us today. And with that said, I'm going to hand it off to our first speaker of the afternoon. Please give a warm welcome

to Burke. So, um I I will admit I uh procrastinated procrastinated making slides for this for quite a while and I I did eventually get a slide deck put together. Um but I kind of decided yesterday, you know what, I'm going to do this whole presentation in the terminal anyway. So, the thing I want to talk about is um how Shopify is using Nyx to power its

monor repo. And uh I'm gonna show you first. I'll go into our monor repo route here. And I'm going to run tech run this thing. And uh here's my slides. So, we're going to talk about tectonics. Um so, Shopify in 2024 decided we were going to be a a monor repo company and this is kind of the story of how that came to be. So, uh I'm

I guess by way of introduction, I'm a principal engineer at Shopify. I've been around for 14 years now and been responsible for most of our generations of developer tools. Um some of you might remember me from Nyxon 2019 in Czecha. Uh I give a talk there about how Shopify was adopting Nyx to power its developer tools. We um we had been kind of gutting our existing kind

of convergent tool. So we have this tool called dev. We did that and we still do actually that operated a little bit like Chef or Puppet. we thought we would replace the internals with Nyx to deal with these problems you start getting to when you have a thousand developers and they can all break their um break their machines in fascinating different ways. Nick solved that problem quite

nicely, but we stopped. this talk is is three three parts. There's there's some context about our journey with Nyx and the Monoro adventure. And then two technical bits, technics and tectonics. So we were hollowing out dev to replace it with Nyx. The the promise is this reproducibility across thousands of The problems we ran into are like Nyx is really hard. Um hard for me. It was hard

for me to learn to build this stuff. It was double hard to teach to everybody in the organization when they would run into the the rough edges with it. Nyx also wants you to go allin. Um, it's very difficult to use Nyx in a kind of peacemeal way or at least uh I was not up for the challenge in 2019 and the organization wasn't really willing to

bite the bullet to go allin on Nyx. I wanted to kind of sprinkle Nyx a little bit to power our development environments in a way that at the time at least I felt like Nyx was fighting me on that. We tried to paper over all the sharp edges and ultimately uh I wasn't able to pull it off, made some kind of strategic errors that annoyed some critical

people and we decided, you know what, we're done with this project. We uh we're going to roll it back. This was this was not the thing to do at the So, was that the wrong decision? I think not really. Um, it was at the time the the effort reward for us was the reward was kind of there, but the effort was so high and we were starting

to get to this point where we were going to have to teach our users more about Nyx and they just wanted to ship their code. Um, it was a pretty high burden to subject them to at the time. So Nyx, as you're all quite aware, has a fairly punishing learning curve, and we didn't feel like there were great resources to teach people Nyx at the So we

put it aside. But then 2024 came around. This is maybe four years after we dropped it. We were sitting around at uh a very large room in Toronto with like something like 6,000 people or something and uh it was hack days so companywide sort of hackathon for for two days chatting with Toby the CEO. Toby is an uh engineer at heart. Uh, and he gets very into

technical details around the company and especially things that are are like very obsessively nerdy such as Nyx. He says to me, "You know what? I really regret killing that NYX project." I like, "Okay, great." Um but by the end of the day we had uh we had decided using this kind of like hack days energy and all the all the sort of ambition in the air you

know what like we actually have to be more ambitious about our developer tools here. What we're going to do is like right now we're deciding that we're readopting Nyx for our development environments. We're going back to local development. And this one is a little bit there's some context here that I'm not going to fully fully fill in. But we had some cloud development environments that led to

these really artisal configurations that it's not super relevant to the story, but dropping that put us closer towards the Nick's dream to kind of and we were becoming a monor repo company. This was arguably actually the biggest of the three things. Um, this is a pretty consequential technical decision and we decided we were going to sort of jump in on that one with both feet and a

sort of consequence we derived from this one is well if we're using Nyx for some things we may as well use Nyx to be the build system for the monor repo as well. So this is the path we took. So why was this a good idea in 2024 but not in 2020? honestly, it's that Claude will write the code for us now. Like this whole thing of

it being a pain in the ass to learn Nyx ny being complex, running into these errors that you just feel sort of stonewalled by and unable to get past. I notice, you know, I I can easily remember the days of feeling this way myself, but I notice it so clearly with with users when they're they're not leaning into these tools is like Nyx has errors that can

feel fairly disempowering, but it sort of doesn't matter anymore. if you toss claude at these things, your problem is solved. They can write the code for you if you don't understand, right? They might not necessarily be able to implement the the the guts of this system for you, but you can certainly craft the system in a way that your users can unblock themselves. Um, the learning curve

now kind of is someone else's problem. So, this is what we basically realized is software is free now. Um, and this isn't just about Nyx, but I do think Nyx disproportionately benefits from this because all of the complexity that made Nyx hard for humans sort of doesn't matter. But all of the benefits that Nyx has always had that, you know, we we sort of adopt Nyx despite

these horrible complexity trade-offs because of these incredible benefits that have still somehow outweighed the cost. The the equation tips so far in Nyx's favor in monor repos also have re relevant advantages here. When all your code is in one place, you can just drop claude or whatever at the root of your monor repo and it as long as you coach it correctly knows everything about everything. Um

you can make crosscutting changes happen in one pull request. There's no negotiating across API boundaries. As long as we're talking about buildtime dependencies, you can atomically change publishers and consumers of a library in one one go. This makes it really easy to do crosscutting refactors in a way that is difficult or more difficult at least in poly repos. There's another thing that has been uh like raise

your hand if if git work trees were a part of your life a year ago, right? A few. Yeah. 20% 25%. Yeah. Like who uses git work trees now? So like 75%. So this is something that um well especially at a monor repo with with agents you you really start wanting work and um I find Nyx lends itself really well to workree style development too if you're

using it for development environments because it depends a lot on how you build your development environments but the way we were doing it before at least had a lot of things um would be sort of wastefully duplicated across work trees. And now with Nyx, this is just a sim link to a thing that you already have in the next store and it takes whatever number of milliseconds.

So this is cool. The dream basically is anywhere in any work tree in any zone. We call them zones. Um fun story about our monor repo. You you want to divide a monor repo up into some kind of package, right? package would be a good word, but Nyx definitely already claimed the word package and we cared about not conflicting with Nyx. Every other good word for what

you might decide to carve up a monor repo into was taken by something important in Shopify. So zones is what we call them. It's fine, I promise. So the Yeah, >> yeah, shoot. Yeah, it's some of this is still so the question is uh Nyx doesn't exactly describe like Nyx is a little bit bring your own build system. How are we doing the build systems for the

various language ecosystems? The answer is um it's it's a little bit ad hoc and we get to punt on this question a little bit because being a predominantly Ruby company like where most of the biggest projects care mostly about Ruby which famously does not have anything resembling a build step, we get to sort of let this crystallize a little bit later in the process than other companies

would. So the sort of philosophy we're trying to apply here um we would like to as much as possible preserve the user experience of the existing ecosystem package managers and wrap and patch them in whatever way it takes to make them collaborate with Nyx to do whatever it seems to be that Um but yeah, the dream is basically we run devop which then calls nyx to do

something and we get instantly a development environment from wherever. So let's talk about how we made that happen. Part two technics. So um our git repo is currently 30 gigabytes. This is the git directory itself. uh even after splitting out everything on import that exceeded some size threshold to be an LFS file. Uh it's fairly polyglot. It's like I I I've mentioned predominantly Ruby, but there is

a whole lot of TypeScript, Rust, Go, some C++, you name it, there's a little bit. Um the thing about a 30 gigabyte git directory and you know over a million files is uh the git index does not love that if all of that is in your checkout. Every operation starts to become miserably slow. Uh a git checkout can take upwards of 20 seconds. Get status might take

seven. Um so we use sparse checkout and we specifically use sparse checkout in cone The only reason we're actually able to stay on Git here is that Microsoft front ran us a little bit on this by making some performance patches to Git a couple years back to make it sort of borderline reasonable for them to put uh Office and Windows in Git. So we've benefited a huge

amount from this and uh we're using Sparse checkout and cone mode. And what cone mode is is basically you specify a subtree route and then everything under that subtree appears in your checkout. Everything that isn't in one of those u patterns does not appear in your checkout. So that works great especially because we sort of carve up our monor repo into these what are the same as

cone shapes called zones. So here's the challenge though. We're Nick based monor repo. You are some code in zone A. You want to evaluate some Nix code that depends on something in zone B. Zone B is not checked out. What do you do? Right? Um you can't just import dot dotbault.net. here here here's the the false path. Our first solution was was fairly hideous. The basic idea

was we're going to use import from derivation to just like call get show to get a file called zone.nix from that zone for every zone that we want to depend on, every build we want to do. It totally worked, but it was Uh, but we kind of built this with a philosophy of like first we're going to make it work. We're going to design the user experience

that we know should be possible. Then we're going to figure out how the heck we're going to make it fast later. Um, so that's what we did and we we limped along with that for for months and it was it was sort of fine. This is actually the code that um that that made that work. And I'm just going to linger on this for a little while

because it's it's hideous. So at the bottom you'll see get show. This was a function that got called kind of indirectly from some other places. And really all it did was get show object to out, right? But you can see the build inputs has worldgget in the middle. What this is is we're going to specifically call git with a git config that passes a git directory of

the prescribed uh clone location of world our repo. Uh but home right this repo the prescribed location is under the user's home directory. And of course Nyx builds run as the Nyx Damon. So we don't know what the home directory is. So, we also had to do this terrible black magic to figure out what the user's home directory is, which was, it turns out if you stat

dev console, you can figure out who is logged in. So, please never do Uh, anyway, that worked. We also actually had to set an ACL on the user's home directory so that Nyx would be able to traverse it. Just don't do this. This is a case study in what to never do. Um, so that worked. But now how do we make it fast? Right. Maybe I I've

heard dynamic derivations maybe sometimes replace IFD. Is that relevant? I don't know. Disable SIP and use fuse. I guess I don't want to make even more enemies in the security team. Maybe we could patch n. So I kind of realized software is free now. So why don't we just patch nicks? Um, and that is what technics became. Um it turned out this was actually really easy. So

what we have now is we have our little fork of nyx where we pass another flag tectonics getitshaw and then we have some new built-ins. In this case one is called I tried to make these very unappealing for our users to call directly. You can see uh anyway unsafe tectonics internal zone source. So we pass it a zone path. Our zone paths start with double slash like

a basel monorreo or whatever. And this gives us um uh basically an accessor to the source. We can pull a file out of it. We can import it much faster, way better. Um and this gives us the imported file. Big improvement. Here are the uh here are the built-ins we added. This doesn't really matter, but just maybe this gives you a sense of like the flavor of

what we did. We have a manifest at the root of the repo that sort of just enumerates all the zones. And so we have that and then an inverted version of it and then a thing that gives us a source given a zone and then some other stuff that doesn't really matter for the purposes of this talk. Actually, one thing that maybe does matter is you'll see

a couple things here like sparse checkout roots and dirty zones. One of the things that Tectonix does is it sort of flexibly decides whether it's going to pull the source of a zone from your local checkout or the git directory based on basically whether there are uncommitted So it tries to be intelligent about building from the local checkout. Yeah, >> freed. >> Yeah. So the question is

um we're just mostly concerned with providing environments. So couldn't we have stopped before Nick's build? So one of the things is that the monor repo is sort of self-describing self uh implementing. So a lot of our tools are actually zones in world. So they are built as builds and then provided as the actual like profile of tools that gets installed as the base layer here. Some of

them come from a lot of them come from Nyx packages, but a lot of them are actually like monor repo builds. So, um, one question that came up actually repeatedly because Toby kept asking me um was why not use flakes? And, um, it's it sounds like a very sensible question like flakes are the thing. Why don't we use them? They seem like they're maybe shaped like this

problem. So, we had two choices basically if we wanted to use flakes. We could put one flake at the root, but then maybe I didn't spend enough time thinking about this, but this doesn't feel like it makes sense to me because you certainly encapsulate all of your inputs nicely, but then you sort of it doesn't feel like you've gained a lot really because you're not solving your

like um your problem. I think there's maybe like sub flakes or something now. Maybe I could have looked into that. The other thing is like if you have one flake per zone that totally feels like it solves this and it kind of does but the problem is then you have every time you want a zone to depend on another zone you have an input registered there and

that has a a hash. So then if you change something that's like deeply transitively dependent on you have spidering hash updates all across the repo possibly including a bunch of files that aren't even in your sparse checkout. So you would end up with especially as you grow it to be a more sort of like deeply inbedded kind of heavily um depended mono repo. You get to this

point where you have a fairly ugly kind of pull request process where you get these these hash updates scattered everywhere. All this on top of the sparse checkout conundrum which like I don't think we'd easily be able to make flakes handle the fact that the flakes they're depending on are just not there. So the the kind of key insight here is we're we're building a trunkbased monor

repo. Everything depends on the concurrent revision. So if you're operating at revision head and you're depending on zone ABC, you are depending on zone ABC at version head and there is no other option. so in this context gets Merkel tree already provides consistency. You don't need a hash to assert that you're you're like maintaining your your version integrity. It's just implied by the fact that you're operating

at the same git commit. flakes would duplicate this. So, I made that case and then Toby pushed back again. He says, "Well, you know, that's fine, but flakes have lazy trees. It's like it's a real shame to miss out on those." And having an engineer CEO is awesome most of the time, but it's really annoying when he's right. And um so then I I I kind of

that that really got to me because it it was annoying. Like lazy trees are excellent and I really wished we could have just had them. But I realized that I could just make them work in Technics and I did. I just opened Claude and I said, "Can you make me have lazy trees?" And it said yes. Um so that was surprising. Um, so yeah, Technics is now

a fork of Nyx and I mean obviously that only worked because the lazy trees code was excellent. Um, so thank you. Uh, but yeah, so Technics is now a fork of Nyx really. It's a set of patches on top of determinate nicks that we just maintain because that's easy too. Um, it has this sort of like monor repo shaw concept. You you know what shaw you're on.

It does all this git object database integration with libgit too which I don't exactly love but git doesn't do a good job of being a library so that's what we're stuck with does a lot of caching to make it all fast and it has lazy trees delightful um I sort of mental model it to myself as like what if flakes but designed exactly for our monor repo

uh it's like it's kind of open source but it won't be directly useful for you but I don't know maybe it inspires something. so then the other part is tectonics. So, um, it used to be the case that I thought of tectonics as two things. The kind of like guts of our monor repo and the other thing I'm about to describe. The guts of the monor repo

sort of got extracted out into technics. And so what Tectonix is pretty much now is the build system, the module system. And there's a fun thing about this word that I just want to want to highlight. Tectonics is a triple entandra, which I'm super proud of because I accidentally woke up in the middle of the night and had this idea in a dream. So So uh yeah,

it's like, you know, monorapo is called world. So it's, you know, tectonic plates, the foundation of world. Also, tecton is the ancient Greek word for builder. It's a build system. And then very prosaically tech to nyx technically makes it a triple on tandra even if it's not a very exciting one. So um it's a module system basically this is we we use the Nexos module system types

lib bunch of stuff but we define our own modules um kind of similar to dev quite honestly um every zone has a zone.next file. This configures everything like languages, dependencies, CI development environment and um everything flows down through progressive refinement. So basically what tectonics does at the end of the day is there's a whole sort of progressively concrete level of options that you can assign that all

bottom out in assigning to an attribute set called targets which is just derivations and everything you could possibly do with tectonics from the sort of outside perspective is pulling out something from the targets attribute set. Um, and that's just a build. So if I would run uh this command, tech, tech is our orchestrator for tectonics. So tectonics is a NYX codebase, but we don't suggest to people

that they literally run the Nyx command to build things. We reexpose this tech command, which has I don't know something like 15 subcomands. But the the relevant ones to point to here that kind of give you the the major mental model are build and run. Um so if I run this tech build areas tools tech default, what this is going to do is it's going to build

the module system for this areas tools tech zone look up the targets attribute set pull an item called default off of it and then run next build on that basically and give you this thing back out. So, an interesting thing about this though is this works. I mean, as I've kind of alluded to, this works regardless of whether or not the zone is checked out because it'll

read directly into the object database using lazy trees. But another thing is it actually works even if world isn't cloned on your disk. And that's a little bit surprising given the mental model I've handed you so far. The way that works is tech actually collaborates with this other other piece of software called Windex. And what Windex does is it's a cache that gets filled by our continuous

build servers that takes this um input of a zone ID. Zones have ids too. It doesn't matter. Just think of it as a zone path. Target name get shot platform identifier and uh it's basically just a key value store that resolves that to a nicktore path. So given this output, we can just try to like next store realize that and presumably fetch it from our binary cache.

So knowing these four facts about what you want which are totally inferable from the um from the command line where I ran tech build you can check this is nice too because it shortcircuits the whole nyx eval step. So now rather than being like two to seven seconds or whatever it's 50 Um so it's filled by continuous build servers. No checkout needed. If it's been built at

that Shaw, Windex has it. So, there are um I made this up for this slide, but I think I stand by it. Uh three principles that I I want to hold to in Tectonics. meeting the user where they are, letting the agent write the the the code in the zone.nix for the most part, and feeding back on varants. So, let's talk about those. So number one, meeting

the user where they are. If you ask a user to like describe a zone, especially like a small project, right? Like if somebody just writes a library and it's like a 100 lines of code and they did it in an hour because it was just sort of extracting a thing. If they describe to you what that thing is, they're going to be like, "It's a Ruby library.

There's nothing special about it." Right? And so your config for that zone should look a lot like it's a Ruby library. There's um or even you know simpler we're playing with a playing around with a few different like highle syntaxes here. The idea is what we want this file to represent is the intent not the in a monor repo. You don't have to lock versions exactly because

you don't have to deal with this whole producer repo consumer repo thing. you have atomic changes across the whole thing and you have CI that can trigger across the entire repo when you change anything. So if bumping next packages breaks your my SQL, you can do this thing where you can actually just leave it fairly loosely specified but then discover um upon attempting to bump that you

now have learned that you no longer had meant just give me my SQL. You now had meant give me a specific version of my SQL. So there's this kind of like retrocausal intent thing going on where you can sort of discover that you actually had meant a different thing prior and now we pin it for you, right? Um, and this gives you an interesting thing because you

now this this sort of like calls forward to another point that I'll get to in a minute, but this gives you an interesting thing where you can sort of like see the increasing complexity of configs and and treat that as a signal that like maybe this project has drifted and there's a sort of reunification of standards that we could do. Um, this is the big one that

that has changed our attitudes about Nyx this time around in 2024 and onward is in um, you know, before humans would write this file and we had to be very careful with managing complexity exposure because we have thousands of users and we don't want to have to make them all learn next. but now basically we can say you can write it if you want but like here

are some claude skills that will configure this project for you however you want just tell it what you want. Complexity doesn't matter or it does but it doesn't matter in the same way. We don't need to teach users nicks. We need to teach agents our module system. And that's a very different problem. So um I'll kind of get to uh one little aspect of that soon. But

the the third principle is feeding back on variance. the complexity that we see in these zone.nix files across our hundreds or thousands of zones is a signal. If there's a file that's just a couple lines, that means we have perfectly captured the pattern that represents what that project is trying to do. If it's high variance, if they have a ton of code, it's suspicious. It could mean

the project is genu genuinely unusual. It could mean we, the tools team, failed to capture a pattern, or it could just mean that user had no idea what they were doing. Um, in any of those three cases, it's probably good to at least attempt to reunify some of that. In some of these cases, you will have the genuinely unusual project and you go, "Okay, this is actually

fine. You're doing something weird and it's good. We're not going to refactor this." But all these other cases are a signal that there's something not captured by the the contract between your build system and your users. But this is all visible, queryable, all in the same repository. And it's very easy to make atomic refactors where you implement a library and refactor the consumer to use it in

the same commit and also prove that nothing has changed. we can also make this queryable and there's an interesting thing where Nixos has this options doc options generation that's really cool. You can also generate um a similar document that includes all of the the current settings and the um definition sites, declaration sites, the intermediate values that are all merged together. This is really cool. Um one thing

that I'm kind of prototyping is what if we evaluate this across all of the zones in the monor repo, compile it into a parquet thing. Parquet for those who don't know is like a columnar kind of data store format. It lets you encode highly normalized, denormalized, I can never remember which one. Repeat all the data, right? And and just like dump it all in in a bunch

of rows and then just query it with SQL. It's just sort of magic. You can say give me all of the zones where it's, you know, Ruby version 342 and higher than one CI parallel parallelism or whatever. Right? So this is kind of cool because now you can actually just generate that file, give Claude a skill to interact with it, and now you can ask questions about

your whole monor repo and have this happen sort of magically, right? What zones depend on the deprecated money library? And that's now just a thing that Claude can figure out locally. a lot of this was was foresight and some of it was a lucky bet. In 2024 when we were sitting around at that company summit like the thing that was motivating this really was we were speculating

about this future and how at some point in the next several years it was likely to be the case that we were going to benefit a lot from AI writing a lot of our code. I think it came faster than we were anticipating. But we feel super lucky or at least I feel super lucky to have kind of made these choices at the time we did because

we're kind of right in this phase right now where we've shaken the snow globe and the snow is falling as we're like trying to figure out how to work with these these new tools at the same time too. So um I think like the the preliminary results for us is I can say like uh Nyx and Monoras both seem to be great choices for this era and

I I definitely see Nyx gaining ground in this ecos I know you've like we've all heard a lot about this over the last day and a half already but it's like it's so true that Nyx is the perfect technology for the moment here. Um there's this effect though where like when when software becomes cheap to write the thing that that starts to matter is taste right like

the the decisions about not can we build it but but like what actually should we build and is this the right way to build it because software is free now. Um, Nyx and our monor are two parts like the critical parts of making this actually true for us like stripping all of the the toil out of our workflow so that we can focus on what we should

build and and um whether it's the right shape. So we're building towards this world where we spin up an agent anywhere in world. Nyx gives it an instant development environment gets us to running the tests instantly. We have situational awareness, no onboarding, no tribal knowledge, no asking someone in Slack. It's just immediately productive. So that's the dream. We're getting there. And Nyx and the Monoro thing have

been critical in this. Yeah. Thank you, Freed. Yeah. So we will have an application zone and that'll be the part that the user is actively working on. It will have in our case often a gem file that will include gem like paths to libraries and those will be when we run devop or bundle install or whatever that orchestrates tectonics to build those into the next store from

it'll read from the git object database unless they're on disk. So you can add them to your sparse checkout and work on them there. But by default it's just like if you had a library that existed somewhere in rubyjems.org work. It's like the source is not super visible until you choose to make it visible to you. >> It's both. Yeah. There there's some sorcery here to just

like make the ergonomics remain familiar while also allowing Nyx to do the nice caching things for us. >> Not really. Um I I think if I had thought seriously about it, I would have suspected that my changes were going to be deep enough surgery that it I would have tripped over it. But it felt so easy to fork it and so easy to maintain a fork that

it didn't seem important. Um, we I'm trying to remember whether we enforce this. We don't have circular dependencies. We at least have not tripped over that yet. I cannot remember off the top of my head whether we enforce that that continues to be true. >> Okay. So, um, if we were if we were not limited. Okay, so the sparse checkout thing is actually funny because I was

just working last night on um the big problem is gits index requires a lot of redundant work um on every command. And I have a a branch that I'm I'm like working on to see like what if this is just a memory mapped file that doesn't require any translation. Can I actually get away with a full dense checkout? So, we might actually be able to pivot to

that if I can figure this out. But, um, the thing that I would do if we did have that is I would get much more paranoid about sandboxing these zones so they can't talk to each other because right now we get to be a little bit lazy about that because the files don't all exist on disk. We are kind of doing a halfway Nyx thing where it's

not the case that everything operates in a Nyx build. Of course, once it goes to our continuous build server and we build certain artifacts, those are done in a real Nyx context and so they won't have a checkout that they can cross bleed across. But um there would be some worry other build systems. We we had actually been trying to use Basil for our especially TypeScript for

about a year before we adopted before we had this sort of 2024 moment and had been largely un unsuccessful with it because uh I think Basil even more than Nyx really insisted on you like basilifying your whole thing at one kind of cut over moment and that was a difficult thing for us to to like a do and b buy into the concept of wanting to Yeah.

And and like I I kind of alluded to this before because Shopify is such a Ruby dominant company, like we get to be a little bit lazier about our build systems, there are ecosystems that are more and less annoyed by this, but we we don't have any like ultra massive compiled projects that need like really significant like distributed build infrastructure. It's possible we will in the future,

but we have kind of some runway to build that as we grow into it. Yeah. So, inter I'm sorry, I've forgotten to repeat so many questions. Interaction with other uh build systems. So yeah, this is all kind of done slightly ad hoc. Like we're trying to use stuff that exists in Nyx packages when we can, but we we are like I think the stuff that exists in

Nyx packages by and large isn't excellent for like live development environments when you want to like convert a cargo toml into like a set of derivations. Um, so like I I hope that the shape that we settle on some of this stuff for can be contributed back in in some way because what we're trying to build, which is all sort of in flight right now, is um

yeah, like a way that sort of flexes from um development um having this like familiar ergonomic and like not really compromising on any major workflows there, but then scales up to a sensible production build um in a way that doesn't really cut corners there either. So, it's it's an interesting thing to try to I think we have time for one more quick question. probably not in one

minute. Uh I I I'll try to encourage the the guys working on this to maybe write a blog post about it. I think that could be could be helpful. Yeah, I think Thank you so much, friends. I think if we had an award for the most Q&A in a session so far, you you win it by a landslide. Um, so we have a break right now and

we will be back at 2:30. So right after the break, we will have steering the future of Nyx OS and Burke will also be back with ILO to do a built-ins talk after. So yeah, enjoy your break. See you soon. I on? Am I on? Okay, I'm on. All right. Hey, everybody. Uh, the MC told me that I'm MCing and therefore I have to actually abide to

the times. So, thank you all for coming back. We're going to go and kick off with a little steering committee and foundation panel. And um I think most of the folks have already spoken today, but we'll do a quick round of intros. Anyways, uh I'm Ron. I'm one of the folks doing stuff on the foundation and also one of the founders of Flocks. And John. >> Hi,

I'm John Ericson. I've been working on Nyx for about six years now. and Nyx packages many more years before that. >> Hi, I'm Philip Taran. I uh have been with Nyx for 2022 2023 somewhere in there. And uh prior to being on the steering committee, I was in the Nyx packages CI that uh successfully got us all the way to you can actually click the merge button

and it'll check your changes and put it in a merge queue. I reviewed the code. I didn't write it. Wolf Gang wrote it. But uh I have uh on a personal note I have uh four kids at home a and uh really enjoy this uh and I work for a distributed file systems company which is tasking me to switch them over to Nyx and is disappointed that

it's taken me a long time because I'm a perfectionist. >> Hello. Uh my name is Julian. You can sometime find find me on the internet under the name Luj. I'm uh my main main occupation is I am a PhD student uh studying the impact of Nyx on software supply chain security and I've been in the Nyx community for about four years I think. >> Hi, I'm uh

Tom Breenney. Uh you might know me as Tom Breck. Uh I've been around in various corners of the ecosystem for about a dozen years. Uh my main focus is that I think that Nyx and things like it kind of give us superpowers to manage software. As it gets more complicated and as software gets more complicated, we're going to need something along these lines. So I am very

focused on larger scale adoption and uh letting everyone know that these superpowers exist and bringing it to them, making it more accessible. >> Sweet. All right. So run of the show how this is going to work. If you don't ask questions, I ask questions. My questions are worse than your questions. So, let's do that. Your questions can also be hard questions. My questions are going to be

lowball, softball questions. So, I'll kick off, but then start raising your hands. Um, and I'm going to be both MCing and answering my own questions, which is always fun, unless you ask the questions. So, to kick off, um, we we kicked off the SC kind of like a year ago. There was a new voting into the SC. Um, you know, awesome people just joined the SC. What's

what's kind of like top of mind? What's what's happening inside of the SC? What's happening inside of Nick's governance from from your purview? Tom, you're holding the mic, so you have to start. Um, yeah. So, I mean, the the first order of business uh clearly was to just kind of, you know, figure out how to work together. Um, that's kind of the the first piece because it's

a, you know, multi-person kind of organization. Um then after that, yep. Then after that you've got um you know kind of the more you know top of- mind things, things along the lines of the uh moderation community teams. Um we also have issues along the lines of like standing up uh next packages team. And then we have like long-standing concerns, right? So we've got focuses on uh

for example, we just talked about things along the lines of trademark uh policy. We're looking at um just like how do we improve the health? How do we improve like funding? How do we bring in money? How do we spend money? What do we spend it on? Um, and so like this is like a large breath of things. Um, I do want to reiterate kind of my

call for action from beforehand is that if there's topics that we are not talking about, please let us know. If there's initiatives or things that you care about, you think that would be like worthwhile to have some discussion or impetus behind or you know some more oomph behind it, then also please let us know. But I'll let uh I guess you guys answer more. >> Yes. So

maybe I can say a few words about the the moderation team situation because it's a topic that was uh dear to my heart at the beginning of the of the term. So um the idea was how we uh recreate some uh some new moderation team and how we make it in a in a way that is going to be sustainable for the future. And the direction we've

been uh we we adopted is to um try to involve some of the old moderators uh that uh have been in the team over the years and and create some kind of of team of uh bootstrap team that that of these people that that are able to discuss and design uh with uh in collaboration with us. what is going to be the future of how we do

uh moderation in the community. And uh so I can say that this this has known some slow but steady progress and we have we are very uh close to uh having this team be announced and and start working on on on the job. Yeah, exactly. Thank you, Philip. >> Yeah. Um, I I think the top of top of- mind thing for me beyond the specifics of the

community team formation and the like this the normal cadence of the NYX community like Nyxcon is where can we help say yes to things that are blocked on this indeterminant no that sort of uh can ambiently hang about the community. Um, I really care about giving permission to to activities that the community engages in and give a clear yes when a clear yes is is called for.

Um, we saw a few of these things where the formation of the NYX packages core team answered to my to my count about 15 open issues that people had fought about in various PRs. I really like that. and I want to do more of that. So, I'm on the lookout for areas that allow us to do more because someone said yes. >> Uh yeah, I think that's

a a great round of issues from the other three members here. I'm not sure I have much to add on on this topic. Um, but yeah, we've we've, you know, I think it's fair to say it was a tense times around the last election, but we've been able to get off to a good note and have a good diversity of opinion within the SC and um, we're

figuring things out. >> Sweet questions. >> Wow. All right, let's go. Yeah. So, I'll just repeat the general question. It's around, you know, what are we doing about security, uh, state actors, malicious intent? There was some stuff happening at GitHub. Um, wants to take that first. >> I got an answer. You got an answer. >> Um, a bit of one. >> Wow, that was like you just

said really yes to that answer. >> We we we have talked a bit about this um both sort of seeing the way the winds are blowing in the world. Um there's some we and and also like in conjunction with binary cash work that was talked about the state of the union. Um so the infot team did a bunch of good stuff there. um that is not only

garbage garbage collecting the binary cache but just like looking what's in it seeing how it matches the Hydra database um and and sort of thinking with you know we we have that we have Hydra and we have GitHub actions and we have of and what are sort of all the bits of infrastructure we have and what's self-hosted and what isn't and what's more loadbearing. So we've we've

have um begun a bit of like a just you know audit is maybe as too strong a word but um going on and we've um seen what's there and we've also um been made aware of some various grant programs around this um from usually uh Europe and um so we're pursuing those two as both like what are our needs but more likely what are our strengths and

how can we help other people deal with these issues. because we're able to bring so much self-sufficiency um that uh other organizations without Nyx would have a hard time getting. Uh in addition to that, uh there's a few things that you've probably seen in Nyx packages itself. The Nyx packages committers repo now has a structured way for people to say yes to new committers, to vet them

in public. Um it's not just one long thread that goes on forever. There's also a standard system for aging um people with commit access out if they haven't made use of it. Uh this is primarily to not to to mean that like if somebody takes over their account, they no longer have the ability to compromise Nyx packages with that commit access. Um and these initiatives by and

large are not taken by the steering committee. They're taken by the Nyx packages core team. And so we're starting to see some of the results of the previous SC's like for my money best decision which is to form a core uh team that was in charge of Nick's packages and a core team which views this contributor access contributor health as one of its primary um uh ways

of of measuring it measuring the health of Nix packages in addition to the security benefits. >> um agree with all that. I want to add one more thing which is um security is not a uh like individual sport right it's a team sport so and it's also not just like okay nyx itself in you know individually you know this community alone and unafraid but it's also us

collaborating with and involved with the entire like software ecosystem and that also means us integrating with them so that's another piece I'm kind of looking forward to is how do we integrate uh like security information so like things along the lines of like sbombs an integration with like you know security vendors and how do we make sure that you know CVE patching is uh reported correctly what

are our processes for that how do we learn about those and then how do we communicate that we have done those sorts of things so uh it's not just like us alone it's also how do we work with everyone else who's kind of looking at and solving similar problems so if you also I'll do my standard refrain if you have any further ideas upon this please talk

to Anyone >> All right, I got another question. Unless you get a question like three All right. What are the biggest gaps right now happening in the next community that we could probably take a look at that we're not What do you guys I can kick off because I I'm I'm trying >> I'm trying to kind of antagonize the group here. >> So, I I've been thinking

about this question a lot, right? It's kind of a weird question like what what can you do that you're not doing that's really important to do? It's kind of like a a mix of a paradigm that you usually don't work against. Uh and I've been thinking a lot about where we have gaps. And I think one of those gaps if you remember my little triangle from today's

session right contributors come first and then you have resources in governments governance and resources is a gap like the size of our community the impact that it has and the fact that we have about 150k of run rate right now is is a joke. Uh right if one of our uh awesome partners decided that they don't want to partner anymore we'll figure it out. But we are

not sustainable for two years. And I think it's very easy to point blame at, you know, the companies outside that are relying on Nix like, hey, you know, what do you think? Open source is free. It's not free. Um, but I think it also has to do with our ability to show them that there's a path and a way to actually interact with us, contribute, and and

actually get involved. Now, I'm not saying that no one's at fault here, but I'm also been thinking a lot about what can I do and what can we do and what can all of the next community do to kind of create more of an industry standard and a path for large and small organizations to get more and more deeply involved especially when they're more reliant on nyx

or have some reliance on ny because the mutual incentives just seem to be there right um I mean we have giant companies right now relying on the binary cash and the NYX Foundation does not have enough money to run the binary cash for more than six months. That is probably not a great situation to be in. Uh again, luckily we have a partner that that helps out

with that. Um but that is not sustainable. So I've been thinking more and more about ways that we can create that standard, create that expectation, and I think it's coming from our ownership to think about it and work it down. Tom's point about if you have ideas, let's think about it. But I've been thinking more about that as we've been going into the grant program, the sponsorship

tiers, and I think there's a lot of effort that we need to do from our end and not just expect money to fall from the sky when we create value. U that's thought I had on the subject. >> I have a reaction to that idea. One of the things I love the most about the Nyx community is our apparent immunity to money. Um, if you're handed money

in general or opportunities of like say Azure credits or the like, what reliably happens is that we don't make use of it and we bumble along and we make use of our home labs and our desktops and so on and so forth. And the ability to make do with a very little amount of hardware and very little um, sponsorship relative to the size of our organization. We're

number we're in the top 10 by GitHub's metrics on almost every like on terms terms of number of contributors in terms of commit velocity in terms of act like uh health and activity. The only thing we're not in the top 10 with is like the ratio of CI minutes per commits. Like, so we're we're really big and we're really used by an enormous number of companies and

yet we're incredibly incredibly frugal for that size. And I actually love that about the next community. And I'm I'm a little bit more loathed, I think, than you are to change that cultural dynamic even if it would mean that you would get a lot of benefits. So for me that doesn't actually fall into the the the bucket of like what am I not doing that I should

do because it's I know the effect that it has on lifestyle to stop being frugal. >> So so I I I agree with you and I'm having we're having this discussion because I think this is the point of like actually an interesting panel and not just us like talking to the either. Um I think my point here is more about how can we have certainty >> right?

I think it's less about oh let's go want to lower the anxiety factor, >> right? It's it's more about how can we know that we can sustain this or promise or make a promise to our ecosystem to our contributors about the fact that we can survive for two years, right? Like if all the lights go off, our generator has two years of fuel. So, it's more about

the treasury aspect. And I think what you're bringing up is really important as a as a cultural standpoint where if we are and hopefully we are able to create these partnerships that that are able to to generate more resources for the for the NYX community, how do we make sure that we keep that almost, you know, startup frugal mindset >> um so that that can continue on

with us regardless of how the dynamics of the cash flow changes. Um so I think that's really powerful. Yeah, I I would say kind of in the middle of these two things definitely we all want to like increase our opex to me that's sort of the worst place to not be frugal but there is some pressing needs on the capex side that I don't I don't think

making like a few key targeted investments is the same thing as just like the hedonic treadmill of like well you know below below our CI is going to be like very expensive perk commit or So I'm I definitely think the foundation has a good um uh good role to play as like the central thing in a community of neutral party not beholden to any one company or

customer or anything to coordinate some of the bigger more abstractly beneficial projects that um uh we can get like a few once and done key wins then that way and the grant program indeed should help a lot with while I do think we we do need to get better at figuring out how to get, you know, some resources and spending them and figuring out where is that

appropriate, where it's not, I do want to highlight though that that is actually a drop in the bucket. Um like compare the the amounts that like that this is we're discussing here and what the like budget is but compare that to essentially all of you like you are the most expensive cost your time your energy your like just add up your attendance to the conference and like

these things start to very quickly dwarf the amounts of money that we're talking about from a foundation level. So part of this is also to make sure that like hey how do we keep encouraging that time that energy right we have a fortunate situation that like our community is actually has lots of energy a lot of incoming interest right there not not a lot of communities have

that asset available to them a lot of them are aging out getting older they're having problems with yeah like they're having problems right and and and we are in the in the fortunate situation of lots of excitement lot of like injection of fresh energy And we have to kind of cultivate that and mentor it and guide it and and and steer it and hopefully like leverage that

to good outcomes. Um >> question Tom was so what can you do? >> Well um I think what are we not doing? I do think we need we need the basics covered. So, I'm kind of a little bit more uh around with kind of you on this idea of we should get those basics covered a little bit better so we have more breathing breathing room. I do

like that. Um I do think that we need a way for there to be a a venue to have larger scale projects executed in a more reliable fashion. that's I think is kind of like the role that uh whether that's something that foundation does whether that's something done externally but I do think we need kind of just ways to execute on larger projects um that also involves

being able to bring in like more participants more organizations and I think that would be helpful uh and the last piece is also making sure that uh we continue to like be welcoming and open to uh incoming that incoming energy so being able to onboard people being able to give them uh like interesting either interesting work, interesting outcomes, uh being able to hand off uh like team

responsibilities and like develop that new like generation of leaders. Um all of that like leads to a healthy organization. Yeah, I I want to say that uh I completely agree with you Tom that um the resource that has the most value in our community is the people and and for me one of the big compass of my action in the Syrian community is try uh to make

the community a more healthy place and try to heal uh existing conflict. We know that there was a lot of conflict within the community in the past few years and I want to try to create the uh for those conflicts to kill and and essentially to to happen more difficultly in in the future. I don't think we can completely avoid conflict but we can try to make

it uh less likely that they will happen. Again, completely agree with everything here. I think that what one thing that I feel that we're not doing right now that maybe is not even in our purview or ability to do, but I I wish to do it. wish to say that like is start to create a community that's kind to each other that treats each other with respect

that uses um what I remember from the Ruby community of um uh we're nice because uh math is nice like that that kind of um way of treating each other even in our diversity in our differences from each other is a thing that I have a sincere ambition to do but I don't know exactly It's very hard to change culture so that that becomes true. Um I

feel myself sort of up against that challenge without knowing how to make forward progress other than being kind in public, being respectful in public and being truthful and direct in public. And figuring out how to navigate that boundary is something I feel as a leader is part of my job. Yep. Justin. I've got three I've got three things. Uh so to repeat the question um there are

young committers uh young contributors to Nick's packages in particular. >> It could also just be new. >> It could be newcomers. That's right. Young young young in contribution history. uh who uh are because they make normal human mistakes are ending up either not making those initial contributions or are getting um immediate negative feedback as their first the first thing that happens and the question is what can

we do to change that culture um so I've seen this culture start to change already uh there's three I think there's three things here the first is there There's a very small set of people who believe it to be their job to police Nyx packages and those people have received private feedback that it is not in fact their job to do so. Um that is a key

component of maintaining a healthy community to directly tell a small set of people who have a misapprehension about what good looks like. So that's number one. Number two is that the contributing guides are constantly in the process of getting shortened. I don't know if you've noticed, but contributing.mmd actually has gotten smaller over time, not a larger, and it's gotten migrated to relevant subsections in different files throughout

the codebase because you don't have to take on the full complexity all at once. And I think the third and most important from my perspective because it's technically able is to move some of those feedback points out of a human being told you to do this and into a machine told you to do this because the emotional impact of the machine telling you to do it is

just so much less. Um, so that's that's my three things. I'm interested if anyone else has uh has thoughts on the matter. >> one thing that I want to say and I think this issue relates also to a much more complicated issue is that we have 9,000 open pull requests on X package and um like it's difficult to for some contributors to take the proper time to

be pedagogic with with newcomers uh when you have that much pull request to review. So I think one thing that was uh working a few years back, I don't know if it's still working, was an automatic label on pull request open by uh first-time contributors. This helps uh that's cool. Okay, so this kind of thing helps to have a special treatment for uh first-time contributors to help

them u a bit more than other p requests. And I think maybe what we need is also try to to um have some people whose job is specifically to try to steer that th those uh new newcomers uh uh steer them into into like uh drafting their first pull request in state that is um that is acceptable for the project. I think it can be very hurtful

if you try to contribute to the project first time you get a a one line two-line review that is like ah change this change change that it can be veryful and and not human. So maybe we need a special role to make those moments more human. Yes, Jonathan. I I'm also like in I don't know the stats for that one. What I do know it may be

a good metric. I want to highlight a different metric that's very very related. So with the advent of the Nyx packages committer um repository uh and the which tracks both newcomers who are saying yes we're going to give you additional rights, additional responsibilities and additional power. We're also at the same time sunsetting some people's commit access who haven't made use of it and are no longer contributing.

I pay really close attention. and I'm subscribed to everything that happens in that repository because I want to see more people being said yes to in the future than there are people who've aged out because they stopped contributing. Um that's my internal like health metric that's related to your um what's my net customer retention uh I think is the the stat you're looking at. Um >> yeah.

Um beyond that though I I there's a there's a diversification moment that's happening in Nyx right now. Like I know there's some people who don't like that diversification moment with licks and determinate nyx and with the um way that the community appears to be perhaps fragmenting. I think this is a necessary step as nyx grows. There will be spaces where upstream nyx and upstream like nyx packages

don't meet your needs and providing a path where you can be in and out is just absolutely necessary. So, I see this kind of um moment as being a good thing for our community and something that actually represents uh people finding a space that's right for them. Uh even though it does come at a complexity cost where you can no longer make some of the guarantees you

could make when there was one unitary implementation where there was one unitary community. I know that's perhaps my take, not our take, but it is my take. Anyone else? >> I I do um actually agree with that a lot too. I think I wrote something similar in my original candidacy document over a year ago. Um yeah, we are always, if you think of like Nyx packages as

some tightly spinning wound up thing, as it grows bigger, as it spins faster, it's going to break apart a little bit. There's going to be other stuff going on as the community gets bigger. And that is, I think, natural and to be expected. Trying to shove everyone into perfect alignment is just not sustainable and and not a hill that anyone should want to die on. So, um,

yeah, I'm I'm quite happy to see there's multiple competing things going on. Um, I mean, I just came from the Nyx BSD talk my co-workers did. Same idea there. like it's good to have multiple kernels, not put all your eggs in a Linux basket or a systemd basket or whatever other basket may be. So I I do think like the the diversity of the members and the

diversity of the technical directions both is very important to keep us on a sustainable resilient path. >> Cool. Yeah. So, so that's that's actually what's beautiful about about Nyx. Um, I think what's beautiful about Nyx and >> yeah, sorry. Um so Jonathan was saying that um external communities have found that if there's not an explicit effort and Jonathan correct me if there's an addition here um to

to maintain some sort of alignment between the diverging efforts that sprout out of one community it becomes they become very siloed and almost impossible or very hard to reemerge right because at the end of the day while that's the whole thesis of open source like you should be empowered to to not agree with the current direction, fork it, take it your direction, and maybe it's the better

one, maybe it's a better one for everyone, maybe it's the better one for you and your neighbor. I don't know. Um, but ideally, if we're able to merge and reduce the redundancy and reuse, right? Um, core basic parts, we can all kind of like the tide can rise all ships. and and like my personal response to that is that I think Nyx has a very unique uh

aspect to it that I don't think exists in in a lot of other open source projects to a certain extent and I think that's Nyx packages. for me, Nyx packages is kind of like the crown jewel. That's that's where all the data is, right? If we're talking about the AI era, like that's the data layer and that's where we all converge and all the usage comes into

and and we all kind of work together there. But then there's different implementations on top of Nyx packages that you can use and and I think that's where we're seeing some of that divergence right now, right? There's like NYX, there's Licks, there's there's a bunch of products and and and services and stuff like that, but at the end of the day, we're all coming back and contributing

down to Nick's packages. So I think that's actually a very fascinating um I I don't know I don't know how intentional it was you know at the beginning but out of the out of the u and where things are today it's actually a a superpower for for Nyx as an open source project and I think why u we're able to get so many contributors because because of

Nick packages >> I I couldn't agree more like Nyx packages is there are so many PhD thesises which describe a world that could exist and then zero people in the world cause that world to exist. There is no necessary through line. That means from the existence of Nyx the tool to Nyx packages the working repository of basically all open source code in the entire world for as

many platforms as anybody can make them work. that that has to occur. That occurs because of your contributions, your effort, your collaboration, your passion and this continued like cycle of building, fixing, maintaining, um, upgrading, automating, toil, reducing, and and so on and so forth. Uh I I believe that NYX packages is the thing that lets Nyx change the world and I uh consider it like Ron to

be the the crown jewel. It's the thing that without which nothing for >> That's about two to three questions. So I'll repeat that. Have we thought about potentially doing a Nixos binary cashmir? Um, so there are some instances of this just kind of already happening naturally, but I think you're asking more along the sense of like how do we explicitly encourage it and set up a >>

Yeah. Um, I I know the idea has been discussed before, but I I don't think it got like driven to a you know conclusion. So, but if that but that's something that you know we could take as something that would reduce risk. it would reduce the kind of reliance upon like existing infrastructure um perhaps reduce costs due to egress. So I I see no reason not to.

It's a good idea. >> Yeah. And I would recommend too I think Jonas and uh Jfly are around here somewhere probably just checking with them. I I also with Tom know for certain that when we had um the migration of the S3 cash sponsorship um we had that like one week I don't know if folks remember it was like I think three years ago now um where

like the entire community got together and we're thinking about a bunch of ideas I know mirroring came up but I I think with Tom here like I don't think we ever saw it through >> yeah really quick um as Tom was saying earlier get in Um, I think this sounds fantastic to everyone on the panel today, but we don't quite know what partner orgs need for this,

especially on a more social than technical level. So, please, if you're in this room or you're watching this online and you have some sort of bootleg mirror going, talk to us. We will be thrilled to bless it and make it totally official. That would be a complete win-win. Thanks, Yeah. Um the question is have we been contacted by or in touch with um the sovereign tech fund

in Germany and other such things in Europe? Uh yes, we have been uh I guess twice now we've applied and gotten a grant from the sovereign text fund. I am administering the grant from the foundation to the various subcontractes and um there's also the uh NGI team um that has a um EU grant. It's also a German grant. Um and we're we're very aware there's typically the

ways things work is less like a dialogue based and more applicationbased which um I have opinions on but um given that's the way it works um we you know we have people that follow this um there's uh there's consortium partners on some of the bigger ones we're we're definitely very involved in pursuing everything that makes sense. >> Cool. Last questions. >> What's up with the minutes on

GitHub? >> Thanks, Halen. Uh Halen Hayen. Halen is asking um what's up with the minutes on GitHub. So >> you you I I can Okay. >> Yeah, we try to we try to publish those minutes uh as fast as possible. Some of them in the last months I guess are unpublished uh because they contain a lot of uh names of people that were in discussion for the

bootstrap team and we will publish those minutes when the bootstrap team is announced. So very soon now I I also have a different a different answer which is much more um uh self-directed. maintaining the minutes to be a lot of work and to be able to go through them and to be able to say uh is this public? Is this should this be public? should this not

be public was it it expended my budget for what I can do and I regret that that is true because I I have a sincere wish that what we do is public and yet I know that some portion of that needs to not be public because that's part of what the job is. Um, I know I've fallen down on some where I was the chairperson, uh, and

just haven't gotten around to, yeah, I need to deal with January. Um, so, uh, that's that's my honest answer. >> I think one of the questions here is is what can be useful, right? Because, uh, the whole idea behind meeting minutes is that we want to make transpar like things as transparent as possible for the folks that care and want to be >> Yeah. And I think

this goes back to the point of like what does the community want from those meeting minutes or from the aspect that we're currently providing it via meeting minutes and are meeting minutes still the right format? Uh does it have to be meeting minutes? Could it be summaries? Could it be you know titles? could be like I think there's there's a potential community discussion about how can we

answer the need of keeping transparency that is not causing you guys to have to add 20% to anything that you do because it's about like removing a name or something like >> Yes. That's that's a that the votes have indeed happened and that is on us to to make that make that be public. Um >> we have we have like >> I have failed my particular uh

campaign pledge of accountability on that respect. I think that that's that's an exactly improvement point and I'm happy we're having this discussion because if the community can tell us what matters at least you guys know that if you got to the P zeros and you didn't fill in the six I don't know the four pages of like redacted names and meeting notes that maybe no one even

reads then maybe those are not necessarily important when you have time for that great when you don't at least we know what was voted on and who voted on it and that's what the community wants and perfect u One back there. >> So I personally agree with you. I think beating minutes are the way we publish them is are pretty useless because they require a bit of

context to be understood. Uh like they are uh the exact word that were pronounced in the meeting which which sometimes is is difficult to understand uh as an outsider. Sorry. >> Where's the question from? >> Ah yes yesment. >> What was the statement? And the statement was uh maybe something interesting could be mi meeting summaries as well and I was going to say that I agree with

that. Unfortunately summaries are even like more workload than just the minutes. >> So ideally in an ideal world we were we are able to sustain this workload but turns out we are not able to sustain the workload of just publishing the minutes and votes on time right now. So it's two step ahead. All right, we're gonna have to wrap up. Um, this is again super useful and

this was just intended to kick off and rekick off these conversations recurringly. We don't need to wait for these kinds of panels to bring things up. Um, write a like, you know, a small note to someone, post it on Discourse, uh, send us some mailing pigeon, I don't know. Um, I had a few other >> an issue. >> Yeah, opening an issue. Um, talk to Daniel. He

knows how to deliver messages via different uh, vehicles. Um, don't do that. But anyways, this was great. Um, upcoming next we have a really cool talk from Ilico on Wom and I think Burke might be joining him. So, we're going to step off. Thank you guys. Have a good rest of the day. And anyone playing Magic the Gathering, I got some cool stuff that I might be

bringing in in about like an hour. So, just find me. All right. Thank you guys. Hello. Hello friends. We have our next session coming up right now. So if you could take your seats, start settling in so we can get started. I think we are all set up and ready to go. So, I'm going to hand it over to our next speakers. Please give a warm welcome

to both Yiloko and Burke. And we're going to be learning about some cool things from them. >> Okay, testing. You can hear me. Okay, thank you. Okay, welcome. So yeah, this talk is about uh uh extending Nyx through the power of web assembly. Uh so this was a collaborative project between uh Shopify and determinate nyx. So the first uh prototype was built by Surma at Shopify who

unfortunately uh couldn't be here today. Uh but Burke volunteered to be not Surma. >> Instead we have me definitely not Surma. You may remember me from 45 minutes ago. >> Yeah. So you're going to uh yeah explain the motivation for W was and uh then I'll give a bit of a demo. Yeah. Yeah. >> So um the the Nyx language right is uh it wasn't designed to

be a general purpose programming language as Elco can confirm. Uh and yet it gets used as one because it's touring complete and therefore of course it gets used for everything. But that doesn't mean it's necessarily performant or maintainable or necessarily an enjoyable experience to push into these corners that it wasn't originally designed for. Um, and the first part of this presentation is a bit of a a

a modern history of of how uh Surma came to prototype web assembly uh in in Nyx. So the motivation was actually this really ties back to the talk I just gave for those of you that saw it. We needed to package JavaScript dependencies that crossed these interzone boundaries in our monor repo and for that we needed to do some parsing of lock files at eval time. So

if you look at like a package JSON, our cargo tumml, this is sort of what we wanted to do where we have um our monor repo zone paths embedded in these files. And so far this is not a big problem. We could parse this at eval time with from JSON or something. But then when you look at the lock files, these are harder. You have in the

best case you have YAML. Sometimes you have a totally custom format that you would have to write a parser for. So in the case of a PNPM lock file, which was one of the things we were using, um the first step was just import from derivation. We're just going to do this the easy way. We're going to make it work first. We're going to make it fast

later. It worked okay. Um but as predicted, you know, as as anyone would predict, this was horrifically slow. um just instantiating next packages, running yq to convert the um the yaml to JSON and then importing it, right? It works, but it's not great. It kind of calls out to be done better. So the first thing Surma thought he would do is well I can just write a

parser for YAML in pure nex because why not? So he u actually took I believe he took the YAML grammar, which is public and it's 130 productions. Ask me how I remember that number. Um, and he decided that he would just throw Claude at it or whatever and ask it to build a test suite and then grind away at a parser until it worked. And he got

pretty close. I think he spent $30 in tokens on it and got to like only 11 failures out of 300. and probably they were 11 that don't matter like the attribute syntax or back references or whatever. But anyway, the the code did not spark joy. So he thought this was probably not the the best path to like properly land on. So from there he kind of started

thinking like what would it take to do this better? So the first thought was well it could be a built-in right there was some discussion on somewhere in the next community about well could we have a from yaml and the thing is yaml is not a perfectly stable language. It does have revisions that come out and um Nyx is unusually sensitive to even subtle variations in these

these things over time. So it seemed like probably not a great idea. Elco was even telling me about a problem with TOML changing a little bit. Yeah, >> function like this would be very hard to uh adopt upstream uh because uh well we we had bad experience with from toml where uh recently we updated the tommo library uh and that caused some very subtle behavior changes I

think in time stamp handling uh and that's very bad because it means that uh depending on what version of nyx you use you get a slightly different evaluation result and that kind of destroys the whole promise. So yeah, that's why uh yeah, upstream we're not very likely to uh to accept uh more functions like that. >> Yeah. And even if we, you know, would have negotiated some

way to be like, okay, well, here's our strict subset of YAML or here's exactly the version of YAML or something like this. There's no way we're getting from Ruby lock file merged into upstream nicks. So that was kind of a non-starter. So from here, Surma thought, well, why not use web assembly? He's actually been using Web Assembly since like way before it was stable. Um, so this

felt like a real cool confluence of things he cares about. So, um, we're going to have like a little brief detour into what is Web Assembly because it's actually extremely cool technology for any of you that aren't really familiar with it. So, the thing that Web Assembly like a funny thing about Web Assembly is it's not really actually either web or assembly. Um, at least not exclusively.

it. What it is really is a really tiny virtual machine specification that gives you this very simple instruction language and a kind of cool set of sandboxing primitives around it and it's uh easily embeddible in a bunch of environments. So to to read from the website here, web assembly is a binary instruction format for a stack based virtual machine. It's designed as a portable compilation target for

programming languages enabling deployment on the web and or for client and server applications. So this is kind of what a web assembly module looks like when it's running. An important thing about this is that basically a web assembly module is a sort of region of instruction code. Um optionally I think a region of linear memory. So you allocate just a little slice of bytes that the web

assembly module can manipulate as it runs. And then it's a set of imported function stubs and exported function stubs. Those are both strongly typed. And the only code that a web assembly module can call are things that the host provides bound to those imported stubs. And the host can optionally call the exported stubs. So what this means is you get strong um sandboxing. You can kind of

probably imagine where we're building towards here. This is kind of convenient for a you know sandboxed execution mode. An interesting thing is you could have kind of uh state leakage across time. If you would instantiate the module once, this linear memory would provide a vector for statefulness. So you just have to also make sure that you don't allow linear memory to persist across invocations and then you

have referential transparency and statelessness. So this is the the way that we can use web assembly modules to provide something that sort of lends itself well to a NYX evaluation time context. So okay we can look at web assembly and say this would be great if we could use it. How do we use it? Right? So web assembly is web assembly. It was meant to run JavaS.

It was meant to run non-JavaScript code in browsers, but um because it's such a an appealing little virtual machine and a fairly simple specification, people have written all kinds of implementations for it. So basically you name a language ecosystem somebody has written a web assembly compiler interpreter jet compiler um something and uh it's now available in in anything and plenty available in C and C++. It's actually

a fun project to write a web assembly interpreter if anybody feels like they would like to get nerd sniped for a couple weeks. Um so the the first thing that um web assembly actually kind of grew out of an earlier project called ASM.js and what ASMJS was this idea of man JavaScript is slow right? Um, what if we could compile C to JavaScript, a restricted subset of

JavaScript, and then have browsers optimize specifically that restricted subset to kind of provide a path to almost have the JavaScript be a facade where you're actually scripting a virtual machine, but it still looks like JavaScript and ships like Jav JavaScript so that browsers that haven't implemented this feature can still run it. So the first compiler written for the to target um you know this this feature was

called mcriptton and it compiled C and C++ to this JavaScript uh subset and actually these days with web assembly mcriptton still is sort of like the premier compiler or at least one of the most well-known ones that compiles C and C++ to web assembly. So this is still a perfectly valid choice. But nowadays there's also a bunch of um Rust actually very competently targets web assembly. There's

more recent compilers for JavaScript, Cotlin, uh C, a bunch of other languages. So these days you can pretty much compile anything with some asterisks maybe to web assembly and you can pretty much embed web assembly in anything. So, it makes it really excellent as a plug-in system because it's actually kind of an anything to anything um plug-in language with strong sandboxing guarantees. And that makes it pretty

interesting as a what if we could put it in Nyx thing. So the thought here then was given all that what if we take this rust uh YAML crate compile it with some minimal wrapper to web assembly and then somehow plug that into Nyx so that um evaluation time code can just call this YAML crate to parse the YAML yield probably a JSON object back and then

be off to the races without any of this uh hundreds of lines of some of the ugliest Nix code you've ever seen to parse YAML and that uh was the the genesis of built-ins.wazom. So Surma wrote a prototype that the presenter notes here say was terrible. I won't comment on whether or not that is true, but uh certainly Elco's version turned out uh much more shippable. >>

Thank you. Uh so those were Sherma's own words and not not mine. Uh yeah so what we did so the initial suras version was basically uh a a a JSON to JSON transformation. So uh uh had a C++ side would had transform a NYX value into JSON then call some web assembly on that and get J a JSON string out parse that and uh uh you'll be

on your way. Uh but that's uh kind of uh inefficient. you can't par pass say uh large values like Nix packages or a Nixos system configuration and do stuff with it. Uh so uh we want a yeah more efficient interface. Um so here's an example of what it looks like uh to do something like yeah writing a uh nyx function uh in this case in rust uh

that calculates the uh end Fibonacci uh number. So there isn't this isn't super exciting, but what all of these functions look like is they transform a value into a value. So this thing capital letter v value uh that type represents a uh opaque uh value on the ny uh heap uh and that can be a lazy value. So it might be unevaluated but we don't care about

that. Uh so it takes one value which could be say an adder set or a list or something arbitrarily big and it's supposed to produce a uh another value. Um so what it does uh in this case uh this value is supposed to be an integer. So we call get int on it. And this is uh a little uh rust helper function that calls uh a wasome

host function. So that causes uh the wasom runtime to uh suspend was execution and call had the nick C++ side to uh check what this value actually is had uh check that it's an integer uh and if so return uh the the actual value of the integer then we do the you know the Fibonacci calculation as normal and then we call another host function to produce a

uh new value on the uh C++ n heap. So this is how you call it. You just say build inwamom. Uh you pass it the path to your uh your your little wom file compiled in or or compiled from whatever language you uh you're using. You you tell it the entry point in this case Fibonacci and you give it uh the argument. So let's uh let's just

prove that that works. Uh well actually let's first uh compare it to uh uh a native uh Nyx Fibonacci. Uh so let's uh take uh well a number that is not too high because otherwise uh it will not finish in time for this demo. So this is the native Fibonacci. And and here is the uh rust was version. There we go. It's a lot faster. So that's

amazing. Uh here's another cool example. We have Mandle brought this of course what everybody needs in Nyx. Oh this is not very helpful. Let's let's do raw output. There we go. Beautiful. Okay. Yeah. I mean I already mentioned the performance but yeah this can be uh yeah orders of magnitude faster uh and very important uh use much less uh memory. So Fibonacci 40 takes 4.5 gigabytes with

the native uh Nyx version because it's yeah this purely uh functional thing that's constantly allocating values and then not freeing them until you uh until your memory gets full and then it triggers a garbage collect. Uh and the W was implementation doesn't do any of that. So >> So it's a little bit faster. >> It's a little bit faster. Yeah. >> All right. So uh but one

important thing uh so Burke already mentioned this but we we of course we want an extension mechanism that ensures the the purity of the Nyx language. So we don't want to enable people to write impure functions in uh in in web assembly. Um so uh rust of course is a is a imperative impure language. So uh this here we have a a function counter uh that uh

uh increments some uh global static uh variable every time you call it and returns the new value. Uh and that would be uh yeah very bad if you could actually do this in Nyx. But with web assembly uh every time you call this function you get a fresh instance of the uh the the web assembly memory. So to see this so here we have this counter function

call it call it and yeah as you can see every time it's reset uh to its uh original state which makes it a little bit less efficient. It would be nice if we could reuse these web assembly instances across calls. Uh but uh the runtime that we use wasn't time has this pre-instantiation feature where it will create a sort of an instance and then we'll do this

copy and write optimization. So uh it's it's pretty fast. you can do something like on this laptop 125,000 was calls per second uh which is a lot less than the number of nyx calls that you can do uh so you typically you want to use w was for uh sort of bigger tasks like uh froml or from yaml which was original use case uh so that of

course now also works Um so uh for example here we have a little uh YAML file and now we can say from YAML YAML and there we go that works and we can do to YAML um yeah beautiful uh and the implementation of this uh I can probably show that um yeah here we go um I mean is pretty short uh So uh this uh uh yeah

this basically this is I won't go to the details but the important thing is uh rust has a YAML uh crate so uh all we have to do is hook that up with uh nyx values and convert between them but all the real world work is already done. Um all right so that's awesome. Uh a very cool demo that uh uh Surma made is uh um uh

executing JavaScript uh in Nyx. So there is this uh quickJS uh uh crate uh for Rust that allows you to uh yeah evaluate JavaScript. Uh so he just hooked that up. So let's see that. So we have this uh little uh little JavaScript program. So now I'm going to do JS eval js test. And there we go. It's done. So yeah, now you can uh in addition

to you you know writing your functions in web assembly, you can also just write them uh in u in JavaScript uh at and completely dynamically uh interpreted. So yeah, that's uh that's another thing you can do. Now, another cool thing, so this morning we had a great talk about Nyx native by by Tom about uh using Nyx as a uh sort of a native build tool. So

a a a build system has essentially a replacement for stuff like make or cmake or or or ninja. uh and there the challenge is that you need to discover uh if you're building something like a C++ project uh for every uh compilation unit so for every top level uh C or C++ file you need to figure out all the uh transitive header file uh dependencies uh because

we want to make every compilation unit its own derivation so we can get nice uh incremental builds. Uh but yeah we need to figure out these dependencies. Um so in that talk uh they used dynamic derivations for that. Uh so here we're using built-ins.wism for that uh which has uh one very nice uh property which is uh yeah that it's not a a derivation. So it doesn't

require copying your source tree uh to the nick store which for something like a a multi- gigabyte uh mono repo uh is is is very important. Um, so yeah, let's just see that in action. Uh, so I'm I'm actually u u doing that. Uh I applied that to a bit of uh Nyx itself. So I'm building lip util. And yeah, that worked because I already compiled this.

Uh but now let's uh modify a file. And now I rebuild. And voila. That's that's done. Um and so one yeah so to show what's happening here. So we have this web assembly function get dependencies that takes a source tree. Uh so I'll do that. Yep. And this is pretty fast. So this is something like 400 source files. Uh so but parsing all those source files and

all the header dependencies transitively. Yeah, that that takes something like 0.1 seconds. So it's pretty fast. Uh and that tells you for every uh um source file uh what its dependencies are. Uh and a a very nice feature here is that it also tells you uh at the granularity of individual source files uh what the uh uh system dependencies are. So for example, we have this one

source file compute levels.cc uh which includes lip CPU ID, but nothing else in Nyx uh includes uh the headers of that uh package. Uh so uh now let's uh uh let's simulate uh that we update Nick's packages and the only thing that we change is that we update uh the lip CPU ID dependency. So we're going to change something here. Um, and now I'm going to rebuild.

Yeah. And now it knows that it only needs to compile uh compute levels.cc and and nothing else. And uh yeah, and here is our library. Uh so yeah, that's uh uh yeah, this is this is very nice, I think. Okay. Um how are we doing with time? Are you okay? >> Okay, great. Um so uh now something completely different. So everything that we had before was about

using web assembly to extend the nyx language. So uh to use it at evaluation time but now that we're linking against this wasn't runtime we may as well use it for other things uh and it was turned out to be something like five minutes of work to add uh platform independent uh uh derivations. So what you can now do is uh have derivations where you set uh

system to wasam 32 wasp p1 uh and you give it a wasam file as the builder um and yeah that will be executed by uh by the wasam time runtime um and yeah so you can have uh derivations that work on any platform. Uh now where is that useful? Uh for example, in things like the Nyx packages bootstrap where at the start of the process uh before

you start you know compiling the C compiler well you need to you need stuff like a shell or uh maybe you need curl to download things. Um and yeah where do you get those from? Well, uh, what Nyx packages does is have a statically a tarable with statically linked pre-ompiled versions of those things for every platform that Nyx packages supports. So if uh if if your particular

platform is not there, then you cannot use Nyx packages there until you figure out how to do uh how to generate those bootstrap binaries. But maybe in the future we could just have a bunch of web assembly uh binaries at the start of the bootstrap process. Uh and yeah solve that problem. Uh and also hey you could imagine stuff like fetchers. So something like a curl download

function uh that that could also be done in uh in web assembly. So yeah there probably lots of other applications. Uh I mean web assembly is just a a great uh uh mechanism for for a plug-in uh system. Um so the status uh so this is currently an experimental feature in determinant nyx. We shipped that last week. Um there's also an upstream PR if you want to

play with that. Uh and we just published a blog post that has all the links uh to uh to all these things if you want to uh uh check it out. And yeah, so we're very interested in your feedback if you want to uh uh yeah, for instance, uh try out writing uh web assembly or nyx functions in other languages than rust because we've really only done

rust. Uh and uh so for instance, one thing that uh we need to figure out is whether our host interface is uh sufficient for uh yeah all the use cases that uh that people might have. So, uh, yeah, we're we're really looking forward to, uh, uh, to your feedback. Uh, and I think that's about it. Do you have anything else to say? >> Nothing to add. >>

All right. And thank you. >> Thank you so much for giving your session. Um, will you be around still if people have questions for you? >> Absolutely. Yeah, perfect. >> Or Yeah. >> All right. How's everyone feeling? >> Yeah, we're almost at the end of Planet Nix. We've got one session left. Yeah, I know. I'm like, where did the time go? I can't believe we're already at

the end. Is everyone having fun learning lots? Yeah, thank you. Um, while we're here, just in case you didn't know, we do have an afterparty coming up, so feel free to sign up for and we're just getting set up for the last session. Michael Stony is going to be talking about AI. I know there's more to it than that, but yeah. >> No, it's at Rockos. Yeah.

>> Yep. 3:45. So, we're just getting set up to get started. So, if you could settle in, get ready for the final talk of the day. And then we get to celebrate a completed Planet Nix together. >> No problem. >> Everything's fine. Everything's awesome. Is there anyone here who it's their first Planet Nyx? Just out of curiosity. Whoa. Yeah, lots of people. Cool. How about second Planet

Nyx? Yeah. Nice. Me, too. Second Planet Nix over here. >> Very fun. >> Yeah, we doing this. All right. So, if everyone could give a warm welcome to Michael Stony, our next speaker and our final speaker. >> Howdy y'all. We apparently order ice cream. So, there's ice cream in the back. And I won't even be mad if you go stand up and get some. So, because I

already had some and it was delicious. All right. So, we're going to talk about nicks and AI. And uh these are topics that have probably been basically plumbed throughout the entire week regardless of what you were doing. So, um but I want to talk about it from a user perspective and that kind of stuff. So, that is way more than one flight at a time. All right,

we're gonna do it this way. Um, so basically AI has every problem that Nyx was built for. And this is really fascinating. Uh, because our AI has all these weird things going on. We have all these LLMs and all this stuff. And we have to figure out what we're going to do with this. We have dependency chaos. We have a lack of reproducibility in the AI world.

In fact, we call that a feature in the AI world. We don't have reproducibility. We have non-deterministic systems now. And that's fascinating. But we still have these problems of how do we run these different models? How do we do training? How do we do inference? All these types And then we have these huge artifacts and these complex setups. And Nyx Nyx was really designed so that you

could get reproducible setups. And that if there was something that was super complex, you could solve it once and have it work in multiple positions, multiple outcomes, multiple platforms, multiple targets. And that's exactly what we want from our AI stacks. So I'm going to be talking about this from a couple different perspectives. In one of these I will be an ML engineer and that is when I'm

wearing this hat and and in others I will be me who's more of a classic release engineer kind of person. Uh and then that I will not be wearing a hat. So dude I did I add props for this talk? You're darn right I So I'm Mike Stony like I said earlier. Um, I run engineering and uh, marketing at Flocks and so I do some different stuff.

I've been doing more marketing lately and that's been interesting, but here we are. Um, and over the years I have been a maintainer of Fedora. I was one of the people that founded Apple. I was a maintainer of Debian. Like I've done all sorts of packaging. I've built I I ported Ruby to AIX. Um, so you know, I've done stupid stuff. Um, and Nyx is something that

I only got into a few years ago when I started working at Flux. I was aware of it its existence. I wasn't that interested in it. Now I'm sad that I didn't learn it a lot earlier, but you know, we live and learn. So my main question was why aren't these ML teams running Nyx? Um because everybody that I talked to that is doing ML, nearly nobody

is running Nyx right basically what it comes down to is, okay, so I read this tutorial and it said I'm supposed to use. Okay, so I'm going to go grab and I'm going to go play with this. And then I figured out how to do something. And then it said I needed to do all this stuff using pip. And then it said that I need to build

everything from source because there's no other way to guarantee how all this stuff works, but I'm a data engineer and I don't know how to build anything. So that's kind of not going to work super well for me. And then it says that on Linux I need to do it this way, but on Windows I need to do it this way, but I'm on WSL, so I

don't even know which ones of those actually apply to me right now. And you just get really, really confused. I have teammates that can train models and their setup doesn't look anything like my setup. And you know what we do? We don't talk about it because it's just scary. I can't figure out what they're doing. They can't figure out what I'm doing. We just have to recite

the incantations from the great old ones and then things will happen. So both of these work. Me, my colleagues, we have we have handcrafted these systems. We have artifacts that are left over from failed installations, virtual environments that are not active, but maybe they're still in the path somehow. We have all of this stuff, but it works. so I don't touch it because I'm a data engineer

and being a system administrator was not my goal in And as a data engineer, I like to point out that I've done a lot of really smart things in my life. So I'm not a dumbass who doesn't understand how to do stuff. It's that I have different incentives than what a lot of people do that are very pro- Nicks. It's not really a skill issue. Like a

lot of these people have PhDs, you know, they're they're very very smart people doing all these ML researches they just go find a tutorial on the internet. I go find a tutorial. My colleague finds a tutorial. Is it the right one? We don't know. We just keep trying stuff until it works. I got it working. I'm not touching it. I'm going to leave it alone. When you

think about it, that's pretty rational behavior. You've done a bunch of complicated stuff. You finally got something to work. You're not really sure why it worked or how it worked, but you did it. And your job isn't to have a reproducible system. Your job is to train a model. Okay? And so every part of the SDLC that has a handoff becomes infinitely more comp complex because I

don't know how it got there. Somebody has handcrafted this thing on their laptop. They've done Python things that are unholy, which in my opinion all Python is unholy, but that's a totally different issue. Um, and so they've done all this stuff and what happens then? Somebody in some operational capacity usually could be a devops team could be classic ops could be infrastructure whomever says all right it's

my job to reverse engineer the incantations that this data engineer has taken so that we can put this into something that resembles production maybe it's a staging fleet maybe it's a production fleet whatever it is I have to do this how many people have reverse engineered a data engineers like stack and tried to figure out what the heck they Yeah, I mean I have and it's it's

it's super fun. Um, and so basically if you have two different data scientists, you have two stacks to start with. If you have three data scientists, you might have nine stacks by then. Somehow they multiply. It's it's a weird math, but it works out. Trust me, I've seen it in real And so this conveyor belt of how you do things across the SDLC ends up being built

out of duct tape where you just you find a solution and you tape it on. you find another solution and you add it in. And so maybe it's a bunch of if statements, maybe it's a shell script that does checking. Whatever it is, it's all these different things. But this is where nyx should shine. I need reproducibility. I need a thing that works on one platform or

another, one target or another. I don't want to have giant models in everybody's home directory. I'd rather have them in a store path where if they're like referenced again, I don't have to install them again. There's all sorts of great things here that we could be doing with Nyx. And so you tell your data team to use Nyx and your data engineer says, "Do you hate me?

Did you just tell no?" Uh, but we're not going to go down that. Uh, so where do I start? Where do I start? If I'm a data I'm supposed to use Nyx. All right. I'm pretty smart. I'm a data engineer. I probably have a PhD or something. And I'm like, they're like, well, first you start with this thing called a derivation. And you're like, cool. I'm going

to go back to my abstract math theory classes and go figure out what that means in this case. And you just start hearing other words, store path, derivation, like all these things that seem pretty foreign to somebody that spent their career training models, perhaps writing some Python, mostly doing math. It's a little weird. Um, do I want to use Nick's develop or Nick's shell or I'm supposed

to do something so that I can see the header files, but I don't know where the header files are, but I need to be able to link against them or compile or something. And so there's just this complex entry point. The shell and the path feel very unfamiliar. And the way that happens and how this exposes is they go to read a tutorial on the internet and

the internet tells them, you know, you're going to type user bin pip or whatever and that's not where pip is when you're using nyx. And so you're very confused already. And so a lot of people, they get to a certain point and they just jump off because they're like, I don't know how to apply this into my life. I'm sure this is brilliant, but my job is

not to sit here and reverse engineer the way this works. Just like you didn't want to reverse engineer the data stack, the data engineer doesn't want to reverse engineer your nicks. So it's it's a birectional problem in some cases. And these tutorials fail in all these subtle ways because of these hard-coded paths or these assumptions that when you're on Windows it looks like this or when you're

on Abuntu it looks like this or they assume that when you say you're on Linux they always assume it's Abuntu for whatever reason. Um and that's the way all these tutorials work. And so the cost of this is that that feedback loop is really expensive. You're trying to cycle, you're trying to iterate. All you wanted to do was run a bunch of inference on a model and

then run evals and test it and see was it, you know, within bounds or not. And instead you're spending all this time on infra. And so then let's say they get that far. I get a data scientist. They get into Nyx. They actually think it's cool. They're using it. and then they want to go use some Python thing that isn't in Nyx packages or that's outdated in

Nyx And that happens, we'll call it 100% of the time. Uh, and so now they have to go figure out how to either get something into Nyx packages or use the Python things that they need without Nyx. And then they get into a religious discussion about purity with their systems team, which again wasn't their goal. their goal is to try and train a model and figure out

if the model's good or not. That's the entire that's the entire workload. Um and so they and then they yeah so these broken guarantees debates all those types of things with impurity like do they just run pip? What do they And so they also need to set up builders and substitutors or whatever if they're going to start building their own packages. And then they eventually realize they

become a release release engineer and not a data engineer and now they're mad at you again. So this is why when they say just use nyx, the data teams often don't succeed or don't choose to do that. Um and their barrier again is not intelligence. It's the workflows. It's the ergonomics. And it doesn't mean the NYX is terrible. I'm just trying to look at this from another

perspective of you want me to change the way I work so that we have this reproducibility, but this tax is pretty high in the cost of my workload. So, there is some good news about some of this stuff. I don't know if you've ever tried to use CUDA with Nyx. It's super fun. Um, Flocks and the Nixos Foundation and Nvidia are working on making that way more

available. Um, it was in a private kind of GA thing for a little bit. It'll be a little better. Um, but we're still working on it because we know that we haven't solved every use case. We haven't solved every edge case. We haven't solved much of the availability of it. if you depending on what commit you're on indict packages and stuff like that. So, still coming. Um,

but at least there's a dialogue going. It started with a bunch of legal documents and now we're actually working on the tech part of it and that's more So, yeah, I just talked about that. Um, but it does prove that some of these barriers can move because I think CUDA being difficult to redistribute, difficult to get going was a longtime problem on basically every Linux distribution. It

wasn't related just to Nyx, but Nyx made it also difficult. Um, and that's getting a lot better. Nvidia has been kind of seeing the power of their reach and wanting to have CUDA succeed more. And that's awesome. One of the things that you want for this reproducibility is that oftentimes training ends up on GPUs, but production inference actually happens on CPUs, which blew my mind when I

learned this because I thought everything ran on GPUs all the time now, but apparently they don't do that all the time. They run a lot of things on once they have the model all trained, they can actually kind of shrink it, optimize it, get it on CPUs, run it cheaper. I guess I don't understand that exactly, but you still need reproducibility of here's how I do the

training and here's how I do the production deployment. And now you want both of those to be analogous appropriately. So the environmental strategy needs to cover reproducibility for the data handoffs across the SDLC for the platform type engineer. Needs to cover reproducibility for GPUs or for other targets that are CPUs. So there's just all sorts of stuff there that is not straightforward from from a user perspective.

And so I have some recommendations on how to handle this. And some of these I would bet this room doesn't like that much. And I'm gonna be okay with that because uh I say stuff people don't like all the time. So I'm pretty used to it. Um one is there's this seam and you can use nyx and have a lot of power but inevitably you will run

into Python packages that are not available that you wish were. And so the way that I've solved this with people actually doing this work is saying if it has a C extension get it from Nyx. If it's pure Python get it from Pi. It mostly works. Um, the last thing I want to do is download giant wheels. Uh, I think wheels are an abomination of a packaging

format. I don't know how else to put it. It makes me very sad. Um, so basically it's not nixify everything. Don't do everything with nyx because I think that that is chasing your tail. Like I just think it's an exercise in futility. I can't mirror all of pi and get it into nyx instantly per commit, per change. Um, it's not just use Docker that isn't going to

get me the Python packages. Like it just doesn't solve that problem. It just shifts it the shape. It puts in a different shoe box. Um, so basically try and have your own system. Have CUDA, PyTorch, binaries, C extensions, you know, all that kind of And then start so basically you own the system native dependencies. And one thing I really like about this is that at least in

a in a nick shell or in a flake or in a flux environment or however you do this, you have your dependencies that are explicit instead of implicit. Because with Python, usually what happens when you try to install something if it's not a wheel and it want needs to link against a library. The way you find out that you needed that library was it fails to install

and you read an error message because there's no way to say I depend on lib XML 2 or whatever it is that it depends on. Um, and it's the same problem with JavaScript or Ruby or whatever else. like that's that's all And so what I've been recommending is UV a lot. Um again, I'm not a super big fan of the Python ecosystem overall, but UV seems to

be the uh least worst of all of the Python package managers, and there are only like 28 of them. So, uh but UV is the one that at least translates into Nyx. I think probably the best currently. Um poetry is decent, but UV is a little better. Um, and so using UV to Nyx, you can actually get more purity there if you have the same set of

dependencies that you keep using over and over again and you don't need to go augment with a different set every day from a different data engineer. So I try to have UV own the pure Python dependencies and trying to have Nyx own the stack that is native and then I get some provenence in there because I have the native stack. If I can trace this with UV

to NYX, then I get the provenence of the rest of the items after a trust on first use setup. Um, which again, trust on first use has its own set of problems. Not going to get into that one right now. Um, and then I have a practical workflow that I can hand to a data engineer and say, "Okay, I'm going to give you a dev shell and

it's got all the C extensions you need. It's got Python at the version that we want you to use. It's got the drivers for the GPUs or whatever. And from here, you can type UV and get what you need." That was the goal. And that has worked. Now the other thing is because I can get the exact things that I want, I can start to optimize this.

Instead of getting this 800 megabyte wheel that has PyTorch in it, I can just go build torch for the actual GPU that I want and package that up for my consumers and now I've saved 75 90% of the disc space that I was going to use otherwise. I find that to be really nice. Some people don't care. Um, but all depends. Are you out of this space?

Are you spend a lot on network transfer? Is it just waiting time of things transferring back and forth? It's also faster. Thanks, Tom. So, um, and so these AI and ML packages need to stay fresh continuously. This is a priority. If I was going to make a recommendation to the Nyx community, it would be how do we keep these these a these Python packages more up to

date? And that includes the rest of the AI stack as well. It's difficult to do. Uh, some of the stuff moves super quickly and I'm super happy when I watched Nick's packages flow. Others of it I you might as well be singing happy birthday to the last time it was touched and it just depends on what package it is. So I I guess I would say that

I would love to see the community adopt away to to push the envelope on Python specifically because of this ML side of things. Yeah, the freshness is highly available as I said. So the better thing is and the other things are UV to Nix I I have had relatively good success with UV to Nix dream to Nix I've had maybe less good but I like the idea

of it so much. Um and right now most of the time those work but they can still be a little better. Um I think they've been kind of in the same state for the last couple years in my experience but maybe they've gotten better. I don't use them every day like I'm not in and out of them all the time. So if I'm wrong on that I'm

and then the final thing is to be friendlier with your ML teams and actually have some empathy and be like, I understand the data you're trying to do or your the workloads you're trying to run. I understand the models you're trying to train. I understand what you're measured on. You're not measured on reproducibility. Like no boss is going to reward that data engineer for having a reproducible

environment. It's going to be the model has a success rate of, you know, above a threshold of some kind. And so as an infrastructure person or as a platform engineer, I have to understand that those incentives are different than mine. But my incentive is to make their work as efficient and effective as possible. And so I'm going to help them with that stack. And so you can

reduce the shelf friction whether again giving them a flake, a dev shell, an environment, whatever you need to do. Give them tools that are handcrafted for them that are reproducible that are used. but you've built them for them, understanding their Build some paved roads and really work on improving the UX. Document things. Um maybe make your own GPT for it. You can do all sorts of things

that can help people. Um but I find that this there's a big barrier between this ML data teams and these platform infrastructure teams right now. Um they don't talk. They don't see each other as human. They call each other idiots. Um you don't understand what I'm trying to accomplish. It reminds me a lot of this like dev and ops thing that we used to talk about for

a long time. I don't know. It rhymes. and so one thing that I was going to bring up earlier originally and then moved it to the end. Why that's important, don't know. Um, is that AI actually could help us here. And that's so 2.5 years ago, I didn't know how to write a nick expression at all. None. Um, every time I looked at Nyx, it I questioned

so many things about my life and I just chose chat GBT. I had a Go project and I said, "Hey, build me a Nix expression to package this up." And Chat TBT said, "Sure, I'll go do that." And then it didn't work. And it said, "You're absolutely right. It doesn't work." And I kept working on it. And it took 31 iterations of me going back and forth

with ChatGpt to get an idiomatic, very simple Go project packaged. Um, and I was like, well, I don't have to worry about this taking my job anytime soon. I did this yesterday. Same package or same Go project. I mean, it's been updated a little bit, but you know, basically the same layout or whatever. And I gave it one prompt and I said, "Make this a Nyx package."

That was it. That was the entire prompt. It built the first time. No notes. Absolutely worked. It was fine. Was a NYX expression exactly the way that somebody probably wanted to see it? It was pretty darn close. Honestly, it was super good. So, um, and so from that maybe the gap between data engineers and platform engineers doesn't need to be as wide because there's a lot of

learning that can kind of be absorbed by other AI tools. Or you can help and say, hey, these are the types of prompts that would work really well for the work you're doing. or I can give you a prompt or I can actually just build a machine that you submit your code and we just spits out the other side of Nick's package and you don't even know

need to know how it works because I'm doing inference on my end. That's great. And so I'm going to summarize this and let you all get go back to ice cream and go to happy hours or whatever it is you all want to do. Talk nicks all night. Uh getting weird. Um so uh but basically if we can keep the Python ecosystem and the AI ecosystem as

fresh as we can in Nyx packages it's going to lower the friction between data engineers and If we can improve Python getting into Nyx through things like UV to Nyx or Dream to Nix or other to Nix type activities that also can make things a lot easier. And if you as a platform type engineer, which is just a proxy for whatever that role is that's doing those

things, can have a little bit of empathy for the way that the data team works, understand their incentives, and meet your users where they are. Think like a product person, understand your users have needs, and deliver them the things they need to do their job. I think we can actually make a lot of headway here, and I'm really excited about the space. So, are we there yet?

>> No, we're not. There's still a lot of gaps to fill. Um but the shape of this answer is starting to become more clear to me and that it's that the gap is closing kind of from both directions where the data the data scientist is getting a little more familiar with needing reproducibility. The platform engineer data science because of all the AI stuff that we're all playing

with and touching all the time. And these LLMs are actually helping us bridge the gap between the two really really nicely um because now they can actually write decent nicks at least for some things. I haven't tried super complicated stuff because I wouldn't know if it was right. I'll be honest. Like I have not tried to package something like KDE, you know. So that would be rough,

I assume. Um, if you want to see this talk, that's where it is. The slides are up. Otherwise, I'm Mike Stony. Uh, thank you very much for coming to Planet Nix. Um, and uh, thank your sponsors as well again. And cheers. Have a great evening,