Ballroom G Saturday Mar. 07 - SCaLE 23x

In this talk, Victor Liaslovski discusses the challenges of organizational transparency in engineering teams, emphasizing that the hardest problems are often not technical but rather related to communication and alignment within teams. He describes his experience working as a tech lead, where he faced repeated frustrations due to a lack of accountability and clarity in project priorities. Victor introduces the concept of 'priority fog,' which refers to the confusion that arises when engineers operate without a clear understanding of their work's importance to broader organizational goals. He advocates for structural transparency, which involves documenting processes, creating visible escalation paths, and making prioritization logic inspectable to mitigate the issues caused by ambiguity. By sharing insights on how to shift from a politeness-driven culture to one of accountability and clarity, he presents actionable strategies for improving collaboration and decision-making in technical environments.

testing. True. All right, welcome everyone to Ballroom G. Just making sure uh y'all are in the right place. Um I'm going to go ahead and introduce uh Victor Lewiski over here for the talk. Um, and so take it away, Victor. Thank you. >> All right. Thank you. Um, welcome to my talk about engineers and transparency. My name is Victor Liaslovski. I've been in tech for over 25

years. I'm currently a principal software engineer at Fleet Device Management. At Fleet, we make endpoint telemetry for corporate security teams and device management for corporate IT teams. So over those 25 years, I've written a lot of code. Once I've written code for 16 hours straight to hit a deadline. I've shipped features. I've fixed outages. I've had great managers. I've had terrible managers. And for a long time,

I thought the hardest problems in engineering were technical. Scaling systems, performance bottlenecks, security flaws. Well, I was wrong. The hard pro the hardest problems were not technical. They were invisible. They were organizational. They were about why we were building something, who it was for, and whether anyone actually agreed on what mattered most. And I didn't realize how damaging that invisibility was until I found myself living in

what felt like a time loop. This is a picture from the 93 movie Groundhog Day. A few years ago, I felt like I was living in one. Same day over and over again. I was a tech lead at a large company. We were working on a project that depended on another upstream engineering team. Every week we had a joint meeting with both teams. Same room, same faces,

same Excel spreadsheet. And every week I would ask the same question. I tried to keep my tone professional, calm, neutral. Hey, Alex, any update on that component we need from your side? And every week Alex would say something like, oh, I meant to get to it, but I got pulled into something else. I'll try to get to it this week. At first, I gave it the benefit

of the doubt. Stuff happens, priorities shift, emergencies come up. We're all adults. But then it kept happening. Two weeks, three weeks, four weeks, same answer, no progress, no accountability, no one stepping in, just polite nods, and then the meeting moving on to the next agenda item. I was boiling inside. I kept thinking, why is no one telling Alex to work on this? Where's our project manager? who's

making the call on what really matters. Deadline slipped, commitments evaporated. Meanwhile, I had to face my boss again with another non-update. And the worst part was that no one was fighting this. No one was arguing. No one was escalating. Everyone everything was polite. Everything was reasonable. And seemingly no progress was being made. That politeness felt mature, professional, civilized. But it was expensive. Sounds good wasn't commitment. Silence

wasn't agreement. A nod wasn't alignment. It felt like we were confusing politeness with alignment. And the cost of that confusion was weeks of delay and months of frustration. In those meetings, no one said this is not a priority. No one said we don't have capacity. No one said another initiative is more important. Instead, we said things like, "Yeah, we'll try." and everyone accepted that as progress. But

we'll try is not a plan. It's a social lubricant. The frustration started eating at me. I'd sit in those meetings and think dark thoughts about Alex, about myself, about the company. Did Alex just not care? Was I failing as a tech lead? Was this just how companies worked? And this wasn't just a few bad weeks. It was years of my life. the same pattern in different teams,

vague priorities, soft commitments, status meetings that produce no decisions. I would go home exhausted, not from hard work, but from the sheer weight of organizational dysfunction. And here's what was awkward. I participated in it. I was polite, too. I didn't push harder. I didn't demand clarity. I accepted ambiguity because it felt socially safer than conflict. The room rewarded smoothness, not truth. And so this loop continued. Looking

back, I can name it now. The cost of politeness and engineering cultures. When we avoid discomfort, we avoid clarity. When we avoid clarity, we avoid decisions. And when we avoid decisions, work drifts. Not because people are incompetent, but because no one is explicitly explicitly saying what matters most. For a long time, I thought the problem was Alex. Maybe he lacked urgency. Maybe he lacked discipline. Maybe he

just didn't care enough. That was the easy explanation. Blame the individual. But eventually, I had to admit I was wrong. This wasn't about motivation. Alex was probably overwhelmed, just like I was, just like everyone else in that room. He wasn't choosing to ignore our request. He was responding to signals I couldn't see. He was optimizing based on information I didn't have. And I was doing the same.

This was not a motivation problem. It was an information problem. We were operating inside what I now call priority fog. A system where everyone is busy, everyone is sincere, and no one can clearly see how their work connects to what actually matters most. In priority fog, engineers guess, managers filter, road maps drift, meetings multiply, and polite alignment theater replaces real decisions. Everyone believes they're doing the right

thing and yet nothing significant moves forward. That was the fog I was living in and it took me years to realize it wasn't inevitable. Let's step back from Alex for a moment because this wasn't just about one meeting. At many organizations, engineers don't know why their work matters. They know what ticket they're working on. They know what sprint they're in. But they don't know how that work

connects to revenue, retention, or strategy. They're executing tasks without visibility into impact. At these organizations, people say things like, "Customers want this." But which customers? Paying customers or prospects? Often no one knows. Other directors I've heard sound strategic, but remain vague. Make it reliable. Make it faster. We need dashboards. add AI. These phrases feel decisive. They feel ambitious, but they don't define tradeoffs. They don't clarify what

should be deprioritized. They don't define what success looks like. They create activity without clarity. Many companies believe they are aligned because they hold meetings about alignment. They believe alignment exists because no one objected. But alignment meetings don't create alignment. Shared context creates alignment. When engineers lack context, they still behave rationally. They improve test coverage. They refactor messy modules. They instrument better telemetry. They harden edge cases. None

of that is wrong. It is responsible engineering, but it may not be the thing that unlocks revenue, prevents customer churn, or supports a strategic customer at a critical moment. When context is unclear, engineers optimize what they can see. That's logical. But companies don't win through local optimization alone. Companies win through global optimization. That means um sometimes shipping something imperfect because it unlocks a large customer or prioritizing

a recurring customer issue over a beautiful refactor or delaying elegance in favor of momentum. Those trade-offs require business context, not just technical judgment. Managers cannot sit over every engineer's shoulder and dictate those trade-offs. Engineering work is knowledge work and it is notoriously difficult to measure. Lines of code, commit counts, velocity points, all of these metrics are imperfect proxies. Managers must rely on engineers to make sound decision

decisions themselves in the moment. But trust without context is fragile. I'll explain. If engineers are expected to own outcomes, they need visibility into what outcomes matter. Otherwise, they're held accountable for goals they cannot see. That's not empowerment. It's structural ambiguity. Looking back at that meeting with Alex, the problem becomes clearer. It wasn't laziness. It wasn't incompetence. It was rational people responding to incomplete information. Everyone was optimizing

locally. Everyone was sincere. And no one had a shared, explicit understanding of what mattered most. Another pattern appears in siloed organizations. Teams cannot see each other's priorities. Each team has its own board, its own backlog, its own emergency. Inside that boundary, everything feels urgent. Everything feels justified. From the inside, the work always makes sense. But urgency is relative. What feels critical inside one team may be marginal

at the company level. Without cross team visibility, teams operate as if their slice of work is the center of gravity. Silos don't just separate code bases, they separate context. Over time, this creates artificial importance. Teams begin to defend their road maps. They protect their commitments. They escalate their blockers not out of malice, but out of incomplete visibility. When no one can see the full picture, everyone assumes

their corner manners matters the most. This is where people start playing a zero- sum game. In non-transparent organizations, priority feels like a limited resource. If one team gains attention, another team must be losing it. So, urgency gets hoarded, language intensifies, everything becomes critical. What would the opposite look like? What would it look like if urgency weren't hoarded, but visible? At my company, Fleet, one of our values

is having short toes. That means not being territorial about work. So each product team has a visible board as shown in an example here. You don't need to read all the details, but it's easy to see if another team is overloaded with critical customer commitments. Uh we can fil filter by customer tags or priority labels. Uh and when visibility exists, work can shift. Capacity can rebalance. Urgency

becomes shared instead of competitive. When teams can see each other's constraints, artificial importance dissolves. The conversation changes from um this is our priority to what is the company priority. That shift is subtle but it is structural. Without shared visibility, cooperation depends on persuasion. With visibility, cooperation becomes rational. Uh some of you may be wondering, what about the managers? It's tempting to assume managers solve this problem. After

all, managers attend more meetings. They see more of the cross team picture. They hear executive priorities. In theory, they can translate all of that context down to the teams. I used to be a manager years ago. A large part of the job was gathering information and relaying it to direct reports. But that relay process is fragile. What if I misunderstood something in the meeting? What if I

didn't get the nuance? What if I missed the full context because I was looking at my email? Managers are human routers. When details must pass through layers of management, it gets compressed and compression removes context. Managers are also saturated with inputs. Even the best communicator cannot transmit everything. Somewhere in that chain, clarity degrades. So if meetings don't scale trust, what does? Next, we move into transparency as

infrastructure. So realizing the importance of transparency changed how I think about leadership. For years I believed alignment came from better communication, more meetings, clear updates, better slide decks. But communication is episodic. Infrastructure is persistent. If clarity depends on who was in the room, then clarity is fragile. Transparency is not culture. It is infrastructure. I call this structural transparency. It means transparency built into the operating system of

the company. Not just information being shared, but documented processes, visible escalation paths, inspectable decision frameworks. Culture is how people behave. Infrastructure is how the system is built. Culture depends on intention. Infrastructure defines defaults. When transparency is treated as a value, we value openness. It remains and feels optional when it is designed into the system. It becomes If context must pass through layers of management, it becomes a

bottleneck. If strategy must be relayed verbally, it becomes distorted. If priorities live in private conversations, alignment is easy to lose. Transparency when designed well functions as infrastructure. That is the difference between conversational transparency and systemic structural transparency. Conversational transparency says I'll explain this to you. Structural transparency says it's already visible. One depends on memory and interpretation. The other depends on design. You cannot scale trust with more

meetings alone. Meetings help. They clarify nuance. They allow debate. But if clarity depends on who attended, the organization resets to priority fog. What scales is not the meeting. What scales is what survives the meeting. If how we operate is written down and accessible, alignment does not depend on attendance. It depends on structure. Defaults are very important. If documentation is optional, it decays. If transparency requires too much

effort, it'll fade. If visibility requires it becomes scarce. Infrastructure is about defaults. Public by default, visible by default, accessible by default. When transparent becomes infrastructure, behavior changes naturally. Teams stop hoarding urgency because urgency is visible. Engineers make better trade-offs because business impact is inspectable. Managers stop being sole routers of information because context no longer depends entirely on them. The system carries the clarity. This is the conceptual

shift. Transparency is not a personality trait of leaders. It's not a vibe. It's not about being nice. It is about designing systems where context flows without distortion and priorities are inspectable without permission. Infrastructure scales, culture follows. And once transparency is designed into the system, something interesting happens. Meetings don't disappear. they improved. Conversations become sharper. Trade-offs become explicit. Disagreements become easier to resolve because the shared state is

already So that was the upgrade I had been missing for years. I kept trying to improve communication. What we needed was to improve organizational architecture and then that's when the fog began to lift. So transparency isn't the same thing as information. Many organizations confuse the two. They assume that if data exists somewhere, transparency exists. But information alone doesn't create clarity. It creates volume. Information answers what happened.

Context answers why does it matter? A dashboard full of metrics is information. Knowing which metrics affect revenue or strategy, that's context. When companies dump raw data into shared drives and call it transparency, they create noise. No does. Noise doesn't create alignment. It creates create it increases cognitive load. Curated context creates clarity that takes effort but it's architectural effort not clerical. It means deciding which question should never

be asked twice and preserving that reasoning behind key decisions. Not just what we choose but why. That why is what lets others act without waiting. In the past that kind of curation was expensive. Today it isn't. With better tooling and increasingly with AI, we can summarize long threads, extract recurring questions, and draft documentation quickly. The barrier isn't cost anymore. It's intent. Mature transparency isn't about dumping everything

into a shared drive. It's about operational relevance. Not every detail needs to be written down, but the frameworks, escalation paths, and priority roles that shape decisions should be visible to the people affected by them. When information is shared without context, people fill in the gaps. Different teams for form different interpretations of the same facts. Volume increases. Alignment doesn't. The reason I bring this up is because organizations

often overcorrect. They swing from secrecy to overload. But volume isn't openness. Relevance is. The goal isn't maximum exposure. It's shared understanding. There's another effect of structural transparency that's often overlooked. It compresses decision latency. In non-transparent systems, decision travel upwards to management for approval and downward for interpretation and sometimes up and down a few more times. Each layer adds delay not because people are slow uh but because

context is scarce. In transparent organizations, more context lives at the edge. Engineers can see which customers are affected. Teams can see which contracts are at risk. Managers can see cross teamam dependencies without constantly requesting status summaries. When visibility increases, fewer questions need to travel upward. That shortens the time between program recognition and action. Transparency does not eliminate hierarchy. It reduces dependency on hierarchy for routine trade-offs and

that reduction lowers latency. In competitive markets, especially now as developers are increasing their velocity with AI agents, latency is more important than ever. The difference between responding in days versus weeks can determine whether a customer renews or churns. There's a final dimension that makes this uh more than a cultural preference. Transparency reduces risk. When priorities are hidden, a revenue risk increases. Teams can spend months building something

that doesn't actually move an important deal forward. And by the time anyone notices, a key customer may already be walking away. There's single point of failure risk. When context lives primarily primarily in conversations, it lives primarily in people's heads. If those people live leave, the reasoning leaves with them. Documentation reduces dependency on individuals. there. Then there's execution risk without shared visibility. Cross team dependencies are discovered late.

Surprises multiply. Firefighting increases. Delivery becomes unpredictable. And there's strategy drift. When trade-offs are implicit instead of explicit, the organization gradually moves in directions no one consciously chose. So looking back at that meeting with Alex, the real risk was not one delayed component. It was systemic drift. Multiple teams optimizing locally, priorities shifting quietly. There was no shared understanding anchoring decisions. Nothing dramatic was happening. There was no outage.

But momentum was dispersing. That kind of drift compounds over time. Transparency is not about comfort. It is about reducing the probability of silent failure. When visibility is structural, risk becomes visible earlier and visible risk is easier to manage. Think about an API. It defines a contract so components don't renegotiate meeting every time they interact. Organizations need the same thing. Organizations need stable, inspectable contracts for how work

moves. At my company Fleet, that leverage does not come from logging every decision. It comes from documenting how the organization operates. Processes should be documented like APIs. We do use ADRs, architectural decision records, and they're useful. This is a sample beginner one, and you don't need to read all the text. Um, they capture meaningful technical trade-offs. They reduce litigation, and they help new engineers understand why the

system looks the way it does, but they're not the structural center of gravity. At Fleet, the core transparency infrastructure is our handbook. You can see the starting page here. You don't need to read it. It is public, so you can view it later. Um, we do not run on Ask your manager. We don't rely on Orange Tradition or Slack Archaeology. We we run on written version processes.

Think about what an API actually provides. It defines inputs and outputs, constraints, and guarantees. It creates a contract that allows independent teams to move without renegotiate meaning every time they interact. Our handbook plays the same role but at the operational level. It defines how hiring works, how releases are shipped, how customer issues are escalated, how product design reviews are conducted, and how recurring tasks are handled. It

defines the contract of how we operate. So when someone asks, "What do we usually do in this situation?" The answer is not in a Slack thread, not in a memory of a past meeting, and not in a quick clarification from a manager. The answer is to open the handbook and follow the documented process. That shift ma matters more than it sounds. Clarity no longer depends on proximity

to leaders or tenure inside the company. It depends on inspectable structure. That is what operational autonomy looks like in practice. Now consider remote work across time zones. Sorry for the busy slide. If you're in another country and asleep when leadership is online, you still have access to the prioritization philosophy, the escalation paths, the release process, and the frameworks that guide decision-making. You're not waiting for interpretation or

context to trickle down to you. Instead, you're following a documented contract that dramatically reduces dependency on real-time clarification and eliminates much of the fog. traditionally created by asynchronous work. So earlier I described priority fog as the condition where context lives in people, processes live in habits and trade-offs lives in human memory. That fragility creates bottlenecks and misunderstandings. When someone leaves the company, context leaves. When someone is

overloaded, clarity stalls. Handbook first design removes that fragility by shifting knowledge into the system itself. The organization carries the reasoning. It's not stored in the IC, the manager or the person who happened to be in the meeting. So at Fleet, our handbook is public. Customers can read it, contributors can read it, candidates can read it, partners can read it before signing an NDA. That reinforces a structural

truth. Transparency is infrastructure, not a vibe. When your operating system is public, it has to be coherent and defensible. Public visibility imposes discipline. Weak processes cannot hide behind ambiguity. If structure is unclear, the world can see it, which pushes us to continuously improve. This model is not free. It requires writing before acting in some cases. It requires updating documentation when reality changes. It exposes weak thinking and

outdated assumptions. It creates friction when something is unclear. But that friction is productive. Instead of vagueness, you get inspectable structure. Instead of dependency, you get autonomy. If you want practical transparency, begin by making how you operate inspectable because processes outlive meetings. But who maintains it? The honest answer is everyone who depends on it. Leaders help define what must be explicit. They shape the operating principles and escalation

path. But individual contributors improve processes, propose edits, and document recurring patterns. The handbook only works if people doing the work keep it accurate. At Fleet, we reinforce that explicitly. We have a public thanks channel where people are regularly acknowledged for improving the handbook or clarifying a process. That might sound small, but it signals something important. Maintaining the operating system is valued work, not invisible labor. Curation is

not clerical clerical work. It is system maintenance. The operating system of a company like any system stays healthy only when everyone treats it as shared infrastructure. What gets curated is not every decision and not every slack debate. You documented decision frameworks, escalation paths, release processes, customer handling standards and the philosophy behind prioritization. In other words, you document how to decide, not every instance of deciding. That distinction

prevents bureaucracy and keeps the system lightweight. Transparency should scale with consequence. High impact patterns deserve structure. What does not get curated are minor tasks, temporary experiments, and low impact implementation details that do not meaningfully alter how the system behaves. A simple filter works well. If the same question is asked twice, document it. If it's a one-off, move on. Filter by impact, risk, and repetition. The goal is

not comprehensive exposure. The goal is preserving reasoning that will predictably recur. To avoid burnout, use templates, adopt a handbook first norm for operational changes, reward computer contributors who improve clarity, and invest in good search tools. AI can assist by summarizing long Slack threads, proposing draft handbook edits, and identifying recurring questions that deserve formal documentation. But AI should not define priorities or strategy. It reduces clerical friction. It

does not replace judgment or The guiding principle is simple. Write for the future teammate, not the present meeting. If an artifact only satisfies today's status update, it will decay quickly. If it enables someone next year to act confidently without asking for permission, it compounds in value. That is structural transparency, not volume, not performance theater, infrastructure that enables independent progress at scale. So we talk a lot in

engineering about technical debt. We understand that when you cut corners, complexity accumulates and eventually you pay interest in the form of slower velocity and fragile systems. Transparency works kind of the same way. Secrecy accumulates confusion. I call this transparency debt. It builds quietly, often invisibly until suddenly everything feels harder than it should be. Transparency debt occurs when processes are known but undocumented. When decisions are made but

not inspectable. When priorities shift but no one explains why. When escalations happen repeatedly but no one captures the pattern in a durable way. Nothing seems catastrophic in the moment. Work continues. Meetings continue. But reasoning slowly decays into memory instead of structure. That interest shows up as repetition. The same questions asked again. The same escalations replayed. The same meeting happening for the fourth time. When I think back

to Alex, I see another angle to our issues and that is accumulated transparency debt. Prioritization logic was not inspectable. Escalation paths were informal. Trade-offs were implicit. The meeting kept replaying because the system had never externalized its reasoning. We were paying interest on invisibility. So to make this practical, here's a simple maturity model for transparency. Not a scorecard, a a diagnostic. Most organizations fall into one of three

levels. The difference here between them isn't intention, it's structure. Level one is the conversational organization. Context lives in meetings and people's heads. Managers act as routers of information. If you miss the meeting, you miss the reasoning. This is where priority fog thrives. Level two is the artifact organization. Context starts living in visible artifacts. handbooks, boards, escalation paths, documented priorities. Alignment depends less on meetings and more on

what people can inspect. Level three is the autonomous organization. The operating system of the company is visible. Engineers can act and reason about trade-offs without waiting for interpretation. Leaders spend less time clarifying processes and more time setting direction. Here's a simple test. Imagine a senior engineer in another time zone asleep during executive discussions. Can they still ship safely, escalate correctly, and explain company priorities without asking their

manager? If yes, you're close to level three. If not, there's still some fog. Some of you may be thinking, "This sounds great, but I'm not the CEO. I don't control the operating system of the company. What can I actually do?" Well, if you want more transparency, the first alignment is upward. You need to understand your manager's priorities. You should aim to know about 90% of what your

manager knows about strategy constraints and trade-offs. Not because you're political, because context drives better decisions. Here are a few examples to uh of things to ask. These are just ideas. If we could only get one thing right this quarter, what would it be? Where do you want us to move faster even if quality dips slightly? What kinds of issues do they want to hear about immediately? And

what does your manager care about most right now? Now, if these questions feel unwelcome, well, that tells you something about the system you're operating in. How do you get to 90%. Here's a few more ideas. Watch meeting recordings if they exist. If not, suggest that key meetings be recorded. Read meeting notes carefully. If none exist, suggest AI notetakers. Use your one-on-one with your manager to clarify strategy,

not just status. Make it a context transfer session, not a task update. Then repeat this pattern. Try to have a skip level meeting with your manager's manager. Trace the signal upward. You don't need to know everything, but if you can see one layer beyond your team, you reduce local opt optimization. Transparency starts with curiosity. Also, you don't need permission to document recurring decisions, write down escalation paths,

move discussions into shared channels, suggest artifacts over threads. You can externalize reasoning inside your scope. That is grassroots grassroots structural transparency. If you understand something others don't, don't hoard it. Share it, write it, link it, surface it. Every time you move context from memory to artifact, you reduce transparency debt. That is how systems change from the inside. So up to this point, I've been talking about systems,

infrastructure, documentation, operating rules. But systems don't just change process. Transparency also changes the human identities of people involved. in a good way. So, I need to admit something. I love being the hero. I love being the person who gets called in late when something critical is broken. I love stepping into a messy situation, untangling the complexity, and saving the day. There's a particular satisfaction in being the

one who understands what others do not, especially when the system is on fire and everyone's looking for answers. I remember diving into production issues and fixing something no one else understood. And then the next the next day people would say thank you for stepping in. That praise feels good. It reinforces a basic human desire. It tells me that I matter. It tells me that my knowledge is

rare and valuable. It tells me maybe that I am indispensable. But there's a problem. That feeling may have been built on asymmetry. Perhaps I knew things others didn't. Perhaps I had context that wasn't written down. Perhaps I had access to decisions that lived in conversations. Hero culture thrives when operating systems are undocumented and context is trapped inside people. When process lives in memory and trade-offs live in

the private threads, the person closest to the information becomes the hero. Over time, after thinking about it, it became clear what was happening. Many emergency rescues reinforce dependence. They rewarded the person with the most context instead of fixing the structure that created the problem. Heroics aren't excellence. They're usually evidence that something structural is missing. When a system needs a rescuer to function, it isn't strong. It's fragile.

These days, I try to work differently. I look for context I need instead of waiting for manager updates. I check the handbook before escalating something that feels urgent. If the same question appears twice, I document it and I move conversations out of direct messages and into shared channels. I'm not trying to be the fastest fixer anymore. I'm trying to make sure the next problem doesn't require a

hero at all. When operating rules are written, when escalation paths are inspectable, and when prioritization philosophy is documented, emergencies actually become less frequent. Not because people suddenly become more disciplined, not because everyone tries harder, but because fewer things depend on hidden context, on something that only certain people know. When reasoning is visible, confusion drops. And confusion is what fuels most chaos. In blackbox systems, information hoarding becomes

power. In transparent systems, visibility becomes power. That shift changes behavior. Instead of heroic rescues, you get reliable execution. Instead of dramatic saves, you get predictable delivery. Instead of late night slack messages, you get fewer surprises. The energy that once went into reacting can now go into designing systems that pre prevent recurrence. When how work gets done is visible. No single person has to interpret reality for everyone

else. Engineers can see escalation paths. They can see prioritization logic. They can see cross teamam constraints. They can also see the actual engineering artifacts, the stories, the architectural documents, the ADRs, the QA test plans that describe how something was tested, as well as the issues that show up when um that show what trade-offs were made. That reduces the need for a human router to mediate every decision.

Instead of asking someone what happened, you can inspect what happened. Fewer bottlenecks means fewer crises. And without constant emergencies, no one needs to play the This does not eliminate complexity. It does not eliminate risk, but it reduces dependence on personality. Reliability replaces heroics. Prevention replaces rescue. Shared visibility replaces information hoarding. And trust grows not because someone saved the day, but because the day did not need saving

in the first place. That is what structural transparency does. It stabilizes the Now there is an ego cost to this shift. Transparency removes mystique. It reduces the power that comes from proximity to hidden details. It exposes weak reasoning and makes dep prioritization visible. Not every engineer enjoys that. When decisions are inspectable and processes are documented, you don't get to feel special just because you know something others

don't. Influence become less about access and more about contribution. Transparency forces an identity shift from indispensable to replaceable by design. That can feel threatening at first, but in healthy systems, replaceable by design is not an insult. It is resilience. When no single person is required to hold the system together, everyone is freer to contribute at a higher level. It's uncomfortable, but it is necessary. Um, there's a

danger here that I need to name clearly. Transparency by itself is not automatically good. If you expose problems, risks and constraints without giving people the authority to act, you don't create empowerment, you create anxiety. Visibility without agency is incomplete. It is pressure without control. There are organizations that share dashboards, metrics, and strategic concerns widely. Engineers could see churn risk. They could see missed deadlines. They could see

customer escalations, but they have no documented decision framework, no clear escalation path, and no permission to move. They're informed, but structurally constrained. That combination easily learned leads to burnout. When you can see the fire, but you're not allowed to touch the hose. That experience is frustrating and exhausting. Visibility must pair with operating rights. Visibility must pair with documented decision frameworks that clarify who can decide what. It

must pair with clear escalation paths so that acting does not feel like overstepping. So at Fleet, one of our values is having short toes. As I mentioned before, that means we're not territorial about work. When priorities are visible and escalation rules are written down, work can move across boundaries without waiting for permission rituals. Visibility increases mobility. It is easy for our people to work on the most

important thing. But if visibility increases and authority does not, we have an issue. When people see the problems but they hesitate, they wait, they second guess whether stepping in will be seen as overstepping. That's a problem. That doesn't feel empowering. It feels like being watched without being trusted. So I say this carefully, visibility without operating rights starts to feel like surveillance. Agency requires structure. Transparency should make

it easier to act responsibly, not just easier to observe what go what goes wrong. So this is where conversation stops about being about tooling and starts being about fairness. We often talk about accountability as a virtue. We expect engineers to own outcomes. We expect managers to take responsibility. But ownership without visibility is structurally unfair. If someone is accountable for a result, they must be able to see

the forces that shape that result. If someone is expected to make sound trade-offs, they might they must have access to the reasoning that defines those trade-offs. Otherwise, we're asking them to guess. Expecting accountability without visibility is unfair. Expecting autonomy without documenting operating rules is unfair. If leadership keeps strategy private, keeps prioritization logic informal and keeps processes implicit, they cannot reasonably demand independent execution. You cannot require responsibility

from people you keep structurally dependent. Um, dependency is not always obvious. It can be subtle. Engineers wait for clarification. Managers become approval bottlenecks. Decisions escalate vertically because the framework for horizontal movement doesn't exist. The organization then misinterprets this behavior as a lack of initiative when in reality it is a lack of inspectable Accountability without visibility and without agency is unfair. When people access to the reasoning behind

those outcomes and the authority to act on it. All right. So what does all of this look like in practice? Let me ground this in something concrete. At Fleek, transparency is not an abstract principle. It is embedded in how we operate day-to-day. It shows up in documentation, in meeting norms, in how we ship releases, and how we escalate issues. It's not perfect, but it is So, one

of our strongest norms we have is handbook first. I mentioned the handbook earlier. If we change how something works operationally, the handbook gets updated before the change is considered live. That forces clarity. It prevents decisions from living only in meetings or Slack threads. If it matters operationally, it must be written down in a way that someone else can follow without interpretation. Also, as I alluded to before,

we operate with open GitHub issues and public boards. Customers can see what we're working on. They can follow the progress of features that affect them. Contributors can see where help is needed. That reduces friction. It also forces prioritization decisions to be visible. When something is not being worked on, that absence of work is inspectable. I can go and figure out myself why something is not being done.

Many of our meetings are public by default, including sprint demos. Those recordings allow customers and community members to see not just what was shipped, but how we reasoned about it. That reduces misinterpretation. It also builds trust. people can see the actual engineers discussing trade-offs, not just the polished release notes. And that goes further internally. Most of our regularly scheduled meetings, including our CEO staff meeting, is recorded

and made available to the entire company. That includes strategic discussions, trade-offs, and areas of uncertainty. When I tell that to uh our interview candidates, they often can't believe it. that changes the informationational gradient inside the company. It means context is not filtered exclusively through management layers. It means I can hear strategy discussions directly instead of relying on compressed summaries. Sometimes I know more than my manager about

a particular situation simply because I watched a different meeting than they did. And here's how I personally consume this content. I'll get on the elliptical machine at the gym, open up a recording, and watch it at 2x speed. If I want to watch another meeting, well, that means I need to continue working out. This also means I don't I don't need to attend every meeting live. In

fact, if I'm not planning to contribute it, it can be more efficient to watch it later at a time that fits my schedule. That flexibility matters in a remote environment. It decouples contacts from attendance. It allows depth depth without blocking real-time work. It also reduces the pressure to be present everywhere just to stay informed. This is structural transparency in practice. Context does not depend on proximity to

leadership. It depends on whether you choose to access it. Internally, we lean towards shared Slack channels instead of direct messages. The norm is to have discussions in public channels whenever possible. That means context is searchable. It means questions answered once can help multiple people. It also reduces information asymmetry over time. We reinforce that structurally. As I mentioned before, we have a public or clarifying a process. That's

intentional. It signals that maintaining the operating system is real work. It's not invisible labor. It is contribution. We also define clear data classification boundaries. Not everything is public. Customer data, security details, financial specifics, those have controlled access. Transparency is not recklessness. It is structured visibility with defined limits. Clarity about what is public and what is restricted is part of the system. And all of this creates agency.

Engineers do not need to wait for a manager to explain how to escalate a customer issue. The path is documented. They don't need to guess how releases are cut. The process is written down. They don't need to rely on hallway updates to understand strategy. They can read the prioritization philosophy. But this model has real friction. There is burnout risk. When information is widely available, people can feel

pressure to consume everything. 80 hours of recorded meetings per week can overwhelm someone new. Search helps, but discipline is required. Transparency does not remove the need for focus. There's also camera discomfort. Some people don't want to be recorded for various reasons. Some people worry about how they come across. That is real. Public visibility can change behavior. Social desiraability bias shows up. People may speak more cautiously when

discussions are Also, maintaining the handbook requires discipline. Documentation drifts if no one updates it. Processes change. Reality moves faster than writing. Someone has to notice when the documented path and the real path diverge. That tension is something we deal with every day. And not everyone prefers this environment. Some people are more comfortable operating with informal influence, with context living conversations, with flexibility that's not written down. Structural

transparency removes some of that ambiguity that can feel constraining to those who benefited from it. So this is not utopia. It is a system with trade-offs, but it is a system designed intentionally. And that intentionality changes behavior in durable ways. There's another pattern that appears when you make systems visible. At first, things actually look worse. When processes are written down, you see inconsistencies. When prioritization logic is

documented, you see conflicts. When boards are public, you see stalled issues. When escalations paths are explicit, you see how often they're used. Visibility reveals what was already there. It just removes the comfort of not seeing it. Hidden context hides dysfunction. It allows teams to believe alignment exists because disagreement was never surfaced. It allows drift to continue quietly. When transparency increases, that quiet drift becomes visible misalignment. That

can feel destabilizing. It can feel like transparency caused the chaos. In reality, it exposed it. You cannot improve what you cannot see. But seeing clearly is uncomfortable. It forces conversations that were previously avoided. It forces prioritization decisions that were previously deferred. It forces leaders to explain reasoning that was that previously lived in private. Structural transparency increases discomfort before it increases performance. That's not a failure. That is

maturity. It is the system acknowledging reality instead of pretending stability exists over time. Fixing weak processes and documenting repeated problems reduces the chaos. Not because you hit it again, but because you addressed it structurally. That's the paradox. Transparency can initially make the organization feel less stable. In the long run, it is what makes stability possible. So when I think back to those meetings with Alex, I don't

feel frustration anymore. I feel clarity. Alex wasn't undermining the project. Alex was rational inside a system that hid its logic. Alex was responding to signals I couldn't see. I was responding to signals Alex couldn't see. We were both optimizing locally because the global picture wasn't visible. Our meeting kept replaying like Groundhog Day because the organizational architecture allowed it to. If the prioritization philosophy had been documented, that

meeting would have gone differently. If escalation rules had been visible, we would have known when to push and when to wait. If cross team capacity had been visible, we would have seen whether delay was overload or indifference. Instead, none of that was written down. So, alignment dependent on politeness, and people carried the pressure themselves. That's the shift. The system failed first. I blamed Alex because I couldn't

see the I couldn't see the system. When operating rules are invisible, friction becomes personal. We start questioning motivation, competence, character. We turn structural ambiguity into interpersonal judgment. But once the system becomes visible, you stop diagnosing people and start diagnosing architecture. The same behavior looks different under inspection. For a long time, I thought leadership meant better communication, more meetings, clearer updates, stronger persuasion. Now, I think leadership is

something else. Leadership is designing clarity into the system. It is building operating rules that are visible, making prioritization logic inspectable, making escalation paths explicit, reducing dependency on personality. Leadership is not the loudest voice in the room. Leadership is architecture. When clarity is structural, behavior shifts. Engineers don't wait for interpretation. Managers stop acting as information bottlenecks. Trade-offs are inspectable instead of political. Escalation feels procedural instead of personal.

You get fewer repeated meetings, fewer heroic interventions, less silent drift. You get adults operating with real agency because the system gives them the context required to act responsibly. That is not cultural magic. That is This is how organizations scale. Not by adding oversight, not by centralizing authority, but by externalizing reasoning. When context lives in artifacts instead of conversations, it survives turnovers. It reduces single points of failure.

It lowers decision latency. It limits ego because influence shift from access to contribution. Clarity redistributes informationational power. And when responsibility and visibility align, accountability becomes fair. That is not softness. That is structural integrity. Priority fog is not a personality flaw. It's not laziness. It's not lack of ownership. It's not a generational issue. It's not a motivation problem. It is architectural. If meetings replay the same ambiguity, that

is architectural. If engineers guess at priorities, that is architectural. If escalation feels political, that is architectural. Priority fog is not fate. It is design. And design can be changed. So if you leave this talk with one thing, let it be this. When your organization feels confused, is it because your people are not trying hard enough? Or is it because the system was never designed to make clarity

possible? Because if it is the system, that is not a character problem. That is a design problem. And design problems can be solved. So before we wrap up, let me pause on this for a moment. These are some of the teams using fleet in scale at scale today. They operate in pretty complex environments and when things get pretty complex uh you need visibility. So many of them

prefer tools that make system state visible instead of hiding it. So that mindset is similar to the transparency we've been talking about Um that's it for this talk. So if these ideas match how you think about your organization, I'd love to hear about your story. Uh here's a few links about fleet. This this takes you to our homepage. Uh and yes, we are hiring. And here's some

information about me and where you can find me and some of my work. Thank you for listening. >> Thank you, Victor, for that talk. I think we have time for one or two questions. I'll take one from this side and one from the other side of the room. jump over here. >> Uh my question was about your handbook >> you know it sounds like it's a very

it's very much a living document that's getting constantly updated, >> right? >> Uh how do you deal with the issue of somebody reading the document maybe a week ago and not catching a new change? How do we constantly keep the organization updated about all the changes? Because I also think that it it seems impractical to notify everyone about every single one of the changes. So, I'm just

curious how that works. >> You're you're right. Uh uh you know, when handbook is updated, if it's like if it's engineering related, it'll be posted in the engineering channel, but of course, you can miss it and later you are doing something. Maybe you're on call and you're doing your tasks and then some task is not getting done. Yeah. So there could be disconnect, right? You're doing something

because you remember the procedure from a month ago but actually there was something else added. So someone will let you know like someone will be like hey why is no one looking at this failure you know tag on call you're supposed to be looking this per handbook. So you know yeah people find out eventually so it's not you know it's not immediate good question thank you >>

thank you anyone from this side >> uh yeah thank you for the presentation it's feels very relevant to sort of struggles that we have at our organization as well um my um I'm curious if there's any friction h having the system operate across techn technical and nontechnical teams. And I'm also curious um what your process is for having consensus on when new things are added to the

handbook. >> Um non-technical and technical. Um I think everyone's basically required to be able to use uh GitHub in our company. so like everyone everyone has to know how to submit submit things to GitHub. So that's like a requirement to work for fleet. Um uh what was the second part of your question? So I'm sorry. >> Yes. Uh how do you achieve um make sure there's consensus

when new things are added to the handbook? Like can one person just add stuff or what's the >> Yeah, we have Yeah, we have PR reviews um and we do have owners for some parts of it. So it's kind of a mixed bag. um you know some some things some areas of the handbook that don't have like an assigned owner then anyone can approve that PR and

it goes in uh of course normally you'd want like you'd let people know at least your manager that you're updating this part uh other parts of the handbook do have an owner so that owner has to approve that PR before it u all right so I'll be outside if you guys want to chat uh thanks everyone for >> you. >> Thank you so much. I guess. Hello.

Oh, good. That's working. All right, welcome everybody. Uh, this is ballroom G just to make sure everyone is geographically situated if they were um planning on going to a different uh presentation, but today we're going to have uh Elizabeth from IBM present. Uh, Elizabeth, take it away. >> Great. Thank you. All right. Well, thank you for coming. I know it's midday, lunchtime and all that. There is,

I think, an hour break after this. Um, so go get food. That's what I'm going to go do. Um, so welcome. Um, I'm going to talk a little bit about open source and closed ecosystems here. U which I will define um in this talk. Um, this is something that's been floating around my head for a few years now. Um, I'm working in mainframes these days. Um, and

that is a very proprietary space. Um, so I've been thinking about ways that we've really made inroads into that space and that's kind of where the talk was developed from. And then I was able to talk to a bunch of people in other industries over the past um, few months to get perspectives from them and weigh in on the way other industries that are more closed are

starting to open up to open source. So what you will see is the culmination of all that and this is the first time I'm giving this talk so it might have some rough edges. Um and I just um welcome anyone after the talk or you know if you have questions please let me know. Um or if there's any ways you can we could improve it or you

have questions from your industry or your organization um that you've seen popping up that I don't cover here I'd love to have a conversation. So welcome. Um so I want to start off by saying I've been to scale a bunch of times. Um so instead of giving you my resume I present to you my resume in the form of scale talks. Um so over the years I

worked for a bunch of different organizations and been part of different companies. Um so when I first um came to scale in 2011 um I was working at a company called Linux Force. We were doing just sort of deployments of like things like lampstacks on Debian and some um sort of more basic things for a tech services provider I was working for out of Philadelphia. Um but

I was very involved with the Ubuntu community at that time. So I I was actually working with Nathan here and other folks um who uh were in the Ubuntu community and so I did a talk on just how you find support in the Ubuntu community. Um I'm also part of a nonprofit in San Francisco called Partemis. Um we've been rather quiet in recent years, but we used

to do a lot of deployments to public charter schools in San Francisco. So I I gave a talk on that. Um and then I went to work for HP was talking more about Ubuntu and then got into sort of code review for systems administration because systems administration is where I'm where I what my actual job had been. Um so we had started to do git ops and

stuff before it really had a name um at HP and in the OpenStack project. Um so I was giving some talks about that. Um I then got into containers with a startup. So I was talking about open source communities and the open source project that was part of that startup. Um and then in 2019 IBM's like hey you want to work on mainframes and I was like

I don't know what a mainframe is so they told me and I learned that they are really cool. Um and so I I joined IBM almost seven years ago um to work work on open source and mainframes. Um, and that's kind of what the my past couple of talks have been around. So, in this talk, I will talk about mainframes a lot, but I'm also going to

branch out into obviously the topic of this talk, which is more generally closed ecosystems. So, as I said, I work on mainframes. That is a Linux one mainframe. That one only runs Linux. Um, and as you can see, they fit into like a 19inch rack spot. They're like seven feet tall and um, very huggable. Um, but I became when I joined IBM, one of the things that

was really important to me was that there was an open source presence. So I joined as technically a developer advocate to talk to other Linux people like myself about like how cool these things are. Um, but one of the things that was important was that we had an open source presence because that was very important to me too. Not just as a Linux CIS admin but as

an open source advocate. So uh I joined the open mainframe project which is part of the Linux foundation and within a couple years I became an ambassador for that project. Um and then inside my own organization when the pandemic hit um that hit developer advocacy very awkwardly because we couldn't go to conferences anymore. Um and so that's when I sort of in my organization I'm like listen

we need like an open source office um that like sort of pulls together a lot of our open source efforts so that when someone in our organization who's doing open source which IBM does a lot of um there's sort of like a central place where they can come and have those discussions and I can redirect them to the right team whatnot. So I I line I proposed

that to my VP in 2022 and she was like yeah let's do it. So, in 2023, I founded the open source program office for IBMZ. Um, and I love this one on the bottom. So, these, it's hard to tell. That's a Lego mainframe, like a little baby one. And then Red Hat came out with like this little tiny Lego set that's like a little like Red Hat

engineer sitting at his little um um thing. So, I put those together and I'm like, "Ah, all the Lego." Um, we also built a life-size um mainframe out of Lego. It's like 250,000 Lego, but they don't let it bring they don't let me bring it with me because it's really big. Um, I also like Lego. Um, okay. So, to get this started, so what what do I

mean when I talk about a closed ecosystem? So the first part of that definition I'd say is it's an ecosystem that like either is mostly using proprietary um software today or like it's it's funny in the mainframe space because mainframe like back in like the 1950s 1960s like everything was open source because software didn't have value. Um so I could sort of say like there was no

propri not much proprietary software at the time. Um, so I always say like like uh mainframe is open source by default and then they're like Liz it's not okay. Um, because things have changed and open source definitions came around and we have a real definition for it. But as you look like through the history of mainframe was like a lot of open source then software became valuable

in the 80s and things became very proprietary and that is sort of the world that I entered into when I joined IBM is that most of the main shops out there were running proprietary code for the most part. Um, or it may be the case that, you know, the company does a lot of development in-house. So, maybe they're using open source components, but they're doing all their

development in-house. So, it's still proprietary. Um, but they have a big tech team working on it, and they're not contributing back to open source. Um, some of these companies that I've worked with, they have dabbled in open source. Um, but what I see this mostly look like is they'll they'll create a product or a project and then they'll just like throw it over the wall, so to

speak, to the community and be like, "Here you go. We gave this to the community. You have to sign a complicated SLA or CLA. You have to that which like gives all rights to your code over to us because we want to stay protected and we'll put a really complicated license on it that may not even be free." Um, and then they wonder why they don't get

community contributions. Um, so they're kind of doing open source, but they don't have real direction in it. And they're so scared of like liability and losing IP and losing control over the project that they don't let anyone else in. And I've seen this a lot. Um, so in this talk I kind of wanted to, you know, take organiz or like I guess industries that fall into these

categories and just pull in some some best practices that I've learned and how to sort of convince industries that are not so friendly to open source to start opening up to it a bit. Um, so of course I'll talk about mainframe quite a lot because that's what I know about. Um, but I've also spoken with people from the motion picture industry in the past few weeks and

also the automotive industry which has some really interesting um um things going on right now. So I broke this down into four key things that a lot of these or uh industries are concerned about. Uh the first is that they will lose competitive advantage. Uh the next one is around security. Security is probably the biggest one that I encounter in the mainframe space all the time because

all the clients that we have are concerned about security. Um the next one is support. Like hey there's no support for open source. Uh I'm like well what decade did you come from? Um um and then they worry about their expertise with good reason because they threw the code over the wall and put a crazy CLA in front of it. Um so they they they're concerned that

they they don't have the expertise to work in an open source realm. the first point here is the competitive advantage. Um what I say here is that may be true. Um you cannot give all of the code in your company and in your industry to open source for the most part. Most industries won't allow you to do that because you do have something that you've built on

top of that that that gives you a competitive edge in the marketplace. So the first thing that an industry or an organization needs to determine is what in their organization makes sense to open source. Um I first encountered this when I was working on OpenStack several years ago where um everyone wanted a private cloud. All the organizations coming together and they're like listen we built things on

top of this. We don't care what the underlying compute infrastructure is and this is the same thing that happened with Kubernetes right like everyone came together to build the core infrastructure because that is not what is the competitive advantage for them. The competitive advantage is what they build on that com compute. So when learning about the motion picture industry, um in their case they focused on like

standardization around formats and colors and tooling and like video formats so that things could be shared between um studios that are working together. Um so it was kind of focused on the tooling and libraries around sort of standardization. Um, and so there's this one that's the the visual effects reference platform. That's a really big part of um the uh motion pictures open source piece is because they

just focus on a lot of a lot of open standards that they can share. So that's where they went with it. In the mainframe industry, um, we pretty much decided that we collaboratively want to make the mainframe easier to use and we want to build up skills. So if you look at the projects within the open mainframe project, most of them are about easing access, sharing scripts

among people of like, hey, this is how you like, you know, add a bunch of users at once and like sharing that sort of tooling. Um, and then we have a big education component to bring in the new generation of folks working on the platform. And then in the automotive industry, I was I watched this video um by one of the leaders of the uh automotive grade

Linux and he was saying that uh you know we you get a new car and then you buy this dashboard sticky thing. You stick your phone onto your dashboard and that is ridiculous. But the problem is like even in a new car the like the the tooling inside the car, the infotainment system is not very good. Um it's I mean I I connect mine with Android Auto

and that's getting there. It's a little better. But um but effectively like we're we're just replacing the tech in the car because the tech in the car is not good. So what the automotive industry decided was like listen like we cannot keep pace with innovation that you're getting on your phone even in these cars because we don't share a common platform. So what they decided to do

was share that common platform. So they created things like automotive um to sort of get like a baseline. So everyone's going to use automate grade Linux and then they don't have to all write their own operating system. Um so the way that these organizations have done it is generally by going to a foundation. Um so first of all the industry sort of decides like you know we

want to work on visual effects and we want to work on standardization or we want to work on a Linux operating system for our cars right you decide what you want to do and then you work to create a foundation. So in the motion picture industry um that was that is the uh academy software foundation and they've gone through a few iterations over the years because they've

been doing this for like 20 years now. Um so the motion picture industry is very much in in open source these days. Um the open mainframe and uh the the ASWF I think that's part of the Linux foundation. Um and then the open mainframe project which again is like IBM and a bunch of mainframe companies coming together to create that under the Linux foundation and then automative

automotive grade Linux that's part of the Linux foundation and then the softwaredefined vehicle which I think is like a working group is part of the Eclipse foundation um so all of these industries kind of went to a foundation and said like please help us out to make the open source happen and that is a very good strategy which I'll talk about more. Um, one of the things

I learned while I was talking to uh, Nithia Ruff, um, she was recently at Amazon. She's worked in OSPOS's throughout her career. Um, but I was really curious to talk to her about her experience at Comcast um, several years ago. And one of the things that she mentioned um, was that standards bodies are a thing that a lot of industries are already used to dealing with. So

there is a parallel to be made sometimes when you're having these discussions about why you need to collaborate with other people like why you have to collaborate with your competitors and they understand standards bodies. So you can start positioning it like it's kind of like a standards bodies for software now that we are in the future and this is really important. Um and they already know the

value of standard bodies. They know if they do not adhere to standard bodies today like they're going to fall behind. Um, and this is one way that has been effective way to approach industries that are a bit more shy about contributing to open source. We can say like it's kind of like a standards body like if you jump don't jump on board, everyone else is going to

have the good stuff and you won't have it, right? Um and then this is one that I have had to develop over the years is once you convince everyone to like start this foundation or start collaborating at least on like a small level um you need to remind your leadership at your organization all the time while you're doing this why you're doing this um because they will

get that bill every year that we're paying the Linux Foundation to do something and maybe it wasn't a good year and they're like we can just not do this and you be like no no no no we need to do that because of all these reasons right so first of all for things like if you think of something like automotive grade Linux right like they're building a

you know they're building upon the Linux kernel and they're doing lots of like lots of really important embedded work to make sure that these things work in cars that is going to save a lot of development effort in house so if you're sort of the person in control of this for your organization or you're really into open source you want to make sure that you're keeping tabs

on what this is saving for your organization. Um, metrics and things. There's lots of open source tooling out there for figuring this out inside of your organization. But making sure you have that like, hey, we are saving money and so the amount that we're paying to the foundation isn't that much, right? Or the amount that we're investing by putting engineers on the open source projects is less

money than we'd be spending on developing in-house and we're getting a better product. So, just making sure that you're able to demonstrate that to your leadership periodically. Um there's one that that came up in the motion picture industry example is that um they tend to have a lot of people that move between studios um because they have a very specific skill set based on the movie that's

being created. Um and having these people relearn tooling based on every single studio is no good. Um, by having consolidated tooling and understanding there's like standards that are open across the industry has been really beneficial to these studios because then they can attract that talent when they need it and they don't have to onboard them with the tools. Like they already are familiar like what tooling you're

using here and like how the color palettes work and all the other movie stuff. Um, so it's one thing that less they have to learn. And again, like if they were not on board with this, like all the other studios would be using the same tools and then you're not, right? So then you can't get that talent over to your organization because they're like, I'm not going

to work for you. You don't use any of the good stuff. Um, or the stuff I'm familiar with. Um, so there's a lot of training time saved. And also just generally your employees, people you hire from the industry, like you know, they're like, "Oh, I use this tool at my old company. I can learn a new one but like or I could not learn a new one

and join a studio that has it. Um and then another one that came up I think it was the automotive industry example is where um there's a really key part of open source like a a really important project and the maintainer has left for whatever reason and now the project abandoned. And what I've seen is that there's these organizations, they will scramble to figure out how to

manage that because they're competitors and they don't have like a neutral place to collaborate. So the problem will be is they're like, "Okay, well, we want to save this project, but I don't want to work with so- and so because like and I don't want to move this to my GitHub repo or I don't want to work, you know, on Ferrari's GitHub repo because I'm Ford, right?"

like um and so like it ends up being like this really problem where like either the project gets forked a bunch of times and that's no good or it just gets abandoned like the companies end up rewriting something internally which is also not great. Um but by having something like a foundation or a working group or some sort of organization that is vendor neutral they can come

together and collaborate there and there's already a known space where they can do this work. Um, and then I will say also just as you are collecting this information and keeping this all in mind, just keep it up to date, like have a document and be like, okay, we're saving this much money and you know, we were able to hire so and so because like they're a

really great engineer and we use the tooling they use. Like just keep a thing open. So when your VP comes to you and says like by the end of the day I need you to validate this expense, which has happened to me before. Um, just make sure you're like ready to be like, "Okay, this is all the stuff we do and this is the value that it

brings." All right, so the big one for me is Um, this one is really funny to me because I remember 20 years ago in 2006, I was working for a company Philadelphia and I love Philly, but they weren't like on the cutting edge of technology in that in that area. Um, so we were still trying to convince them about open source and I I feel like I

I can dust off my old decks from 2006 now with these organizations that I'm encountering in the mainframe space because they're saying the same things. They're like, "Oh, if the source code's out there, can't anyone just write a vulnerability?" And I'm like, "Oh my gosh, guys, there have been books written about this." Um, and so it's like for part of this is kind of just like, you

know, dusting off those old arguments and being like, "Okay, that's not actually true. There's been research and studies and like all kinds of stuff done to show that like you know for the most part the the core open source projects are um um better security-wise than than some of their proprietary counterparts. Um but the good thing also is that in the past 20 years open source has

gotten so much better with security as well. So I remember when I was coming here at scale maybe eight years ago. Um, one of the things I was talking about is adding testing to your open source project and that was kind of new. So, um, I'm glad everyone took my advice and added testing to their open source projects because it's basically ubiquitous now. Like if your open

source project doesn't have tests, you are kind of falling behind. Um, so open source projects have been implementing testing and increasingly like security is part of those tests. Um, in some of like the Linux Foundation projects, you can't even graduate as a project until you pass like have certain security badges and have certain security scans in your open source project. Um, so there's been like a lot

of, you know, proactive work being done in this space to make sure that these projects are more secure than they were 10 years ago. Um, and that's just being incorporated in a lot of their automation, which is pretty cool. Um, another thing that's been really helpful is that more huge companies are involved in open source. Um, and one of the benefits here is like, you know, IBM

like before we adopt a piece of open source software or before we um, incorporate it into a product or give it to a client at all, we have a whole host of testing that we do on that software. Um, and then if we find problems, we contribute those to the open source project. So take you know IBM and multiply that by the dozens of companies who are

big and have massive testing infrastructures right so now you have all these companies who are not only running the tests that are being run in the open source project they're running their own tests to make it like enterprise ready um and this has been a huge thing for open source because now we're getting experts from the actual from the industry um from major companies who are working

on the software to give um back to those projects through those that testing Um we've also been um like devoting security engineers from our companies to the security boards on these projects. So oftentimes you'll have if there's like a vulnerability that comes into a project, it can get embargoed if it's a security vulnerability and then it's looked at by security experts from various companies who have come

in to contribute to open source and be on this security panel. And part of that is the self-interest, right? Like that means the company is in on the embargo. like they get to know what's coming down the pipeline security-wise, but then they also have their security experts who can actually work on remediation. Um, so there's a a a compelling reason on both sides to have those security

experts in the projects. Um, and as I said, like these days there is an established mechanism for most projects at least the larger ones to report security vulnerabilities. Uh because one of the concerns that I hear is like, "Oh, what if someone finds a bug or a security vulnerability in your software? They submit an issue and now everyone can see that and then we've got a zero

day that everyone knows about." I'm like, "Okay, well, don't do that." When you submit the issue, you have to go to the security team, right? Like the there is a mechanism in place for most open source projects now to report security vulnerabilities um for the big ones. Um and then again the foundations like the um the ASF, the Apache um foundation, they have a security team that

will actually guide their member projects through issues. Like if a C a CVE comes up for a project in the Apache Foundation um the ASF security team will jump on that and be like hey what do you need from us? Like we will help you through this so we can remediate this and move on. Um, the Linux Foundation, their projects are, since they do things like the

Linux kernel, which is a very different beast from a lot of open source projects, they they give guidance on how to sub submit the security um uh issues to the project. So, like if you're not sure how to submit it, the the Linux Foundation can help you out. Um and they so they they also and and they also have like direct support for those projects like they

they provide um like scanners and other tooling to members of the Linux Foundation so they can do their own scanning and find those vulnerabilities before they even have to be reported. Um we've also had some very bad things happen in the open source world security-wise. Um and thankfully we we had the right response. um you know things like like heartbleleed when that hit that was that was

a huge huge wakeup call. Um and so one of the things that happened out of that was the the core infrastructure initiative was founded in 2014 and part of that was making sure we had funding for some of these like core open source utilities that everyone is using at the heart of their organization. Um, so that got funding from a bunch of major companies in the industry

um to say like yes, hey, like we want to make sure that OpenSSL is secure. So we're going to give money um to make sure that that happens and fund engineers um to do extra security to make sure that this is the most secure thing that could possibly be because everyone uses it. And then I say recent, but XZ is what two years old now. Um but

that was, you know, the the the uh the back door that was put in through social engineering. you've got someone who is a trusted member of the project and they shouldn't have been trusted and that has opened the the eyes of the community a lot more to the trust that needs to happen in open source communities. So I was actually just saw a talk on Thursday about

one of the trust mechanisms that's being put into place um to sort of trust users and and communities um to try to build up um against the sort of social engineering that happened in that situation. Um, and it's it's a shame that huge incidents like this is what has to wake us up to this. And I think as technologists, most of us knew these hap these things

happened, but to convince our bosses something major had to happen, the ones who have the money. Um, but but it's going in the right direction. I'm really happy to see open source um taking these things seriously and the companies investing it in. Um and then today, so if you Google the core infrastructure initiative, you're like, Liz, you said that was created and it was awesome, but now

it's gone. Um it was actually just pulled into the the broader um open source security foundation. Um I've been to a few of their events and the thing I love about the open source security foundation is they develop not only like best practices for open source software, they have a badging program for pro projects, um but they also develop tooling. Um, and it's really like a home

for tooling um that is very uh both like like scanning for for vulnerabilities but also just like it's a home for security software um for open source projects. Um and so they help in general and then have conferences around all this stuff and like best practices and are continually developing things for open source projects to follow. So we are in such a better place security-wise than we

were even you know five or 10 years Um the other thing I hear a lot about in mainframe space is the support component. Um a lot of these companies are concerned that they like they're using open source software. They just downloaded it from the internet and no one's going to support me. Um first of all I'll say that's a very outdated view of things. Um there are

a lot of companies out there who are doing open source software support these days. Um, which is it just makes this myth just increasingly untrue. Like it's just it's not the case anymore that there's no one out there to support the software. Um, but if you do find yourself in a space where you're like, "Hey, my company wants to use this. There doesn't seem to be a

company behind it. Um, or there's a company but they don't offer it for like, you know, working in mainframe like they don't offer it for my platform or I don't think they have the right tooling around what I need for this." Um but they don't Um they just look at a company's website and they're like ah they don't support us and they move on. Um so my

my plea is to say like just ask like you know email someone on their sales team um or you know go higher than that and say like hey you know I've got this potential big support contract coming your way like can you can you support us? Um, and additionally like a lot of these companies who are offering support, they tend to have a stack that they want

to sell you and you can get like support for a specific product, but if you find that like you want support for a specific product and you're using something else that you really can't find support for, that company may have a solution for you. So really just like engage with the teams at those companies and say like, "Hey, I need support. I want to pay for it."

Um, and just see what you can we can get out of that. And you get further than you might expect. I will say um and if that doesn't work um you could also approach larger companies who you may have had business dealings with. So I like working at IBM I know we have clients who come to us for everything even though we don't do everything. So what

we have is we have an extensive program with our um like independent software vendors that we are partnered with and so say someone comes to us for you know support on some piece of open source software we may say like okay but you have to go to this other company for that. And in some cases, we actually have gone to that other company that supports it and

be like, "Hey, like can you add S390X support? We're going to be your partner in this. We're going to help you do that." And then you'll get this customer. Um, and like having having a customer ready for that development work is like the biggest thing for us. So again, like you know, come to your IBM or come to your whoever you're working with in the technology space

and see if they have influence over getting support for that piece of software that you're looking for or that maybe they'll support it themselves. but it's just not as big as a problem as I think a lot of people make it out to The second part to this is really just taking a step back and asking yourself, do I need a support contract? Um, this came up

at a panel I was doing um back in October around open source software and this one guy was mentioning that like the new engineers that they're hiring on their teams, they don't want the support contract. They know how to use Google. They know how to use stack exchange. Um, and these days like they know how to take a pile of documentation and shove it into AI and

get answers out of it. So like it the support contracts are kind of a thing of a bygone era for some of these organizations and they just go to them by default. But I think a lot of companies need to, you know, step back and say like is this really important for me? Do I actually need a support contract or is my tech team capable of figuring

this out on the figuring out themselves? Um, another thing I learned from from speaking with someone um was that like sometimes they'll get a support contract for a year and then they'll be like actually we now have our expertise in house so we don't need that anymore or like the support contract made us feel better but it turns out we don't actually need it so they'll just

drop it after a year once they build up that in-house expertise. Um so yeah just asking yourself like do I actually need this today? And the last thing I want to dive in rather extensively to is is expertise. Um, so again, companies are afraid that they don't know how to do open source. Um, especially if they're not necessarily a technology company. Um, I mean, look at the

automotive industry. I'd say they're a technology company, but a lot of them are like, "No, we make cars. We don't make software." I'm like, well, a lot of you make a lot of software, but essentially they're just like, we make cars. I'm like, all right, all right, fine. Um, but there are a lot of companies, you know, we make widgets, we don't make software, right? So, they

just don't believe that they have the expertise to do something like contribute to open source software. Um, so my my first thing I' I'd say to them is like you don't have to know how like there are foundations out there that will do this for you. Um, and they kind of one of my friends referred to them like foundation in a box. like they will give you

all these resources which I'll talk about. So the big ones of course there's like the Linux Foundation, the ASF um there's other ones like Eclipse and other ones out there um that do very similar things and also there are like industry specific areas. So like if you're in finance or if you're in energy there's like sub areas where you can look for foundations that will support your

journey in open source. Um but you just so first you see like you know which foundation aligns with your goals and then you're kind of set like they will help you through this whole thing. So they will provide things like proven governance structures. So they'll set you up with like say like okay you need a technical steering committee technical steering committee maybe you need a board of

like executives. essentially they'll just lay out what the options are and you can kind of like work with others in your in your industry to figure out um what makes sense um and sort of pick and choose and make decisions. Um they'll also provide you with technology frameworks. So like if you need hosting or if you need code or if you need like all the other pieces

that come together mailing list in an open source project they will provide that for you. Um, and then they can also help with like growing your community. Um, which is I think a lot of um, a lot of folks struggle with because like you know you're working in the motion picture industry, you don't know how to build an open source software community. You've never worked out there

in the open with folks. Um, and so the that's one of the things the foundation can help you do is like be more open sourcy about how you approach your projects. Um, and then for your, you know, contributors, there's a lot of stuff inside. So there's mentorship programs, there's scholarship programs, there's diversity and inclusion programs that the foundations have expertise in administering and then they can use

that expertise to give it out to your community and and benefit your contributors. So it's it's just having worked on open source projects where like I founded them and like I created my own little fifom like it's it's like going to a foundation is such a refreshing experience because I don't have to cobble together all that stuff myself. Um, another big one for companies is again they

are so scared about licensing and intellectual property. Um and so one of the things that I saw when I was working on OpenStack was that when that all came together um with the member initial member companies um like HP where I was working like they contributed lawyers to like work with the foundation um to put together all the legal frameworks to make sure that all the member

companies felt satisfied with the open source licenses that were being used with like the agreements that were being put in place and everything was all like put together in a way that the enterprises were happy with. Um, and that's something that the foundation can facilitate because your organization, you know, your Ferrari may not want to work with Ford's lawyers, right? Like, um, but if there's an intermediary

of the foundation, they can get them together, um, to work through those it also means that as I mentioned earlier, like that means you're not throwing the software over the wall and just letting it sit there and not being able to accept, you know, recruit contributors. It really helps having that firm foundation from experts who have already done it before. Um, and it protects you as well

because you're finally like your lawyers are like, "Okay, we can do this. We can contribute open source with our competitors." And that's a really good feeling when the lawyers say it's okay because they never say it's okay to me. Um, so anyway, so I I've sort of talked in vagaries here, but just to give you an example of a project that I've I've worked on. Um, so

I run the software discovery tool, which is really just like a NodeJS like web app with a database backend. It's nothing special. Um, but it's mainframe related. Um, and so when I said like we need this because we need to be able to search what's on open source softwares out there. Um, the Linux Foundation helped me and the Open Mainframe project with like they created the project

for me. Like they they gave me a logo. Like I I don't design logos. You've seen my slides, right? Um, so like they made us a logo and they like gave us little options like how about this? How about that? I'm like, "Oh, that one's cute." Um, they gave us um a production hosting environment. So we have like a virtual machine devoted to the project that I

don't have to pay for, which is nice. Um, we're hosted in the open mainframe project GitHub organization. Um, they set us up with a channel on Slack. They set up us up with a mailing list. There's a whole like meeting and calendaring system that we can use that keeps recordings and does transcriptions of our meetings. Um, and then we also have access to like the tools um,

for scanning and things. Um, and they regularly send us reports that they run centrally to say like, hey, like you're in violation of this license, or oh, you're all good, or like you have this one piece of dependency that's that you need to fix up because it's got a security problem. Um they will also periodically come to the technical steering committee for the open mainframe project as

a whole and do like mini training sessions to remind us um like what um badges are available through the Linux Foundation and what training is available for free for open source projects to make sure that that security um things are being checked off on the lists. Um so they've been helping our project with some of that stuff. Um we've also leveraged their um paid mentorships and this

is one that I found incredibly valuable. Um so the mentorships are are financed by like the member companies. So the companies that invested in the open mainframe project um they fund those mentorships but it's kind of like they pay a bunch to the open mainframe project and mentorship is one of the places where it goes to. Um but essentially you get a student for like 12 weeks

um over the northern hemisphere summer um and they will work on like a specific project inside your organization. Um but the reason that this was so valuable to me is because working on mainframes this weird niche little place that no student ever knows about. Um we were listed with all the other Linux Foundation mentorship projects. So there's like hundreds of projects for every session and we were

just in that list with everyone else. So people were able to discover our mentorships without me having to like tell every student in the world that mainframe still exists which was a relief. Um so then we ended up getting like dozens of people applying for our things who had never heard of mainframe but are like hey let's I'll work on this. So that's been that's been huge

for us. Um and those mentors that I the mentees that I've had through the program like some of them have gone on to speak at conferences and get great jobs around the world. So I'm just I'm so proud of them. Um, the Linux Foundation has also helped our project get on things like they put us out on their blog which gets fed into the Linux Foundation promotion

machine. Um, and also like podcasts and interview style things. Um, even speaking slots at at conferences like Linux Foundation events and and like the open mainframe summit. Um, and just making sure that you're you're getting the the out there um as much as you want. We've definitely leveraged all of that. And this is all stuff that I could have cobbled together myself, but it was really nice

that I didn't have to. Um, and it's it's really nice having that community there um of support even if I know what I'm doing, right? Um, the other thing that that came up a lot in my conversation is um open- source program offices. Um, so I remember being at an open source summit a couple years ago and there was a woman who was working at Ford and

she was like like she's like my team does not understand open source and I think they were trying to develop an open source program office at the time. Um, and essentially what an open source program office does is they they coordinate um the uh like the the goings on of open source within your company. Um, and it's kind of like they they handle like how uh people

in your company contribute, um, what licenses it's okay to contribute to, and they just sort of make sure they keep on tabs of like who's contributing what, and that your contributions are going in a way that properly reflects the company, right? Um, and so there's this thing called the to-do group, and that's what that QR code is there. It's just I think it's toddroup.org. Um, maybe. Um,

yeah, to-doggroup.org. And that is like expertise from dozens of people working in OSO all over the world for a bunch of different companies in a bunch of different industries putting together their collective knowledge so that if your organization wants to create an OSO they can go to to-do group and basically like learn everything on how to create one and what is necessary and what kind of expertise

you need in that. Um I run an ospo by myself. I would not recommend that but the benefit for me is that I I have like broader IBM above me. So I run the OSPO for IBMZ specifically. Um but I have like IBM above me like does all the legal stuff and contribution guidelines and stuff. So I don't have to worry about that. I mostly work about

around like coordinating open source project which is the fun part. Um but there's a lot of resources. So there's like they wrote they wrote a book um like a PDF that you can download about about running ospos. And then there's the the OSPO definition there too. If my definition wasn't good enough which it I think it was sufficient but you can dig deeper. Um, the other thing,

um, one thing I was talking to Nithia about about her experience at Comcast, um, was that the, uh, your developers may not know how to do open source yet either, especially if they're coming from more traditional like waterfall development workflows or where they feel like they need to land a whole feature in one patch. That does not go over so well in open source communities. Um, so

there's also guidelines for how you contribute to open source um that are out there. And again, to-do group made a great one that the Linux Foundation that's in this QR code here. Um, it's an open source guide just to like tell you how to open how to contribute to open source. Um, and then there's lots of guides out there like GitHub has some good guides that cover

like the technical parts of how to contribute, not necessarily the social components. Um, so there's just guides all over the place. And the really key thing is that you make sure that your developers in your organization get a hold of these guides so they can build up their expertise and then also doing like routine training internally or telling them to go to a conference um and meet

other people um who are doing open source to sort of learn what the culture is because we do have a very distinct culture in the open source world and if someone tries to land a 10,00 line patch in my project I'm going to not be happy about it. No, I'm going to be happy, but I'm going to tell them that that's not how we do it very

nicely, and I'm going tell them to break it up into like 10 patches. Um, but things like that, like companies just don't necessarily know that, right? Because if that's how they do things internally, that's how they're going to approach open source projects. But I'm not going to review a thousand lines of code. No, thank you. Um, another thing that was that was came out when I was

talking about this talk was the uh just in general like organizational reputation. Like there are companies in tech that we want to work for. Like they have a good reputation. We know they have a good tool chain. They work on interesting things and you want your organization to be one of those companies. Um, I know we're going through like a kind of dip like you know it's

cyclical, right? Right now a lot of people are looking for work but I'm sure we'll come to a time very soon where companies are fighting over engineers again because we always go back and forth. Um but I think you know you want your company to be desirable to developers and a lot of developers these days want to work on open source. Um so having that opportunity and

making sure that your organization is you know your developers are being trained in a way that is open source friendly um is really um helpful to to attracting the that that top talent um that you want in your So all right got time for questions. So just just to conclude my checklist is just you know identify what you can open source in your industry um to make

um and make a foundation out of that. Um and then continuously address security concerns. Um continuously address and foster that support concern that that pops up in in a lot of a lot of companies and then you know support building open source um expertise right there in your organization. So, some references and how to get a hold of me. That is my cat sitting on 3D printed

components of a typewriter that I'm building. And if you want to talk about that, that'd be fun later. Okay. Um, but yeah, any questions? Anybody in the audience have any questions for Elizabeth? Great talk, by the way. >> Yeah. Um, I will start on this side. Anyone from this side since I'm already over here? Anyone? Anyone on this side that wants to ask a question? >> Oh,

there's one over here. >> Back. >> Where? Oh, over there. Okay, perfect. >> Thank you very much for sharing your reflections of two decades. Uh, it's really interesting to see how you navigate and through all these uh different projects you contribute. Uh what are the key uh lessons for fresh for example who is like right now uh there are there is a community base from the developer

side which was not ready I think in the in the past years. So uh it looks like from your presentation that you have been part of many projects. So what is your take about uh specific uh sticking to one open source project and make it happen u and sticking to it uh as opposed to hopping many projects. So what is your recommendations for uh uh developers out

there who are in the market uh who want to make a difference apart from their 9 to5 jobs. um how should uh they what mistakes they should not make? Yeah, that is a very very good question because I I noticed you know you're you're paid to do a job and your job is working on that open source project like what happens when you want to move on

from that and I I think I think I've left a little piece of myself in every open source project I've worked in and I to some extent I have somewhat stayed involved in that community or at least offered a gentle handoff of the work that I was doing because the fact is like people understand that people change jobs. Um they know that like open source stuff is

is you know you do move on from things. Um but I will say just like being aware that when you join an open source project you become responsible for something. And just because your job changes doesn't mean that responsibility goes away. And so I've tried as hard as I can like when I'm transitioning jobs I'd be like listen guys like I have to go work on this

other thing now and like now it's time for me to hand off like here is what I did in the project and this is what you're going to need to replace and those transitions when you're upfront about it and also like I didn't I didn't fly to Mars right like you can still reach out to me and a lot of the work I did in OpenStack people

were still messaging me like a year later and I'm like oh yeah yeah sure I I've even like hopped on poll requests and things like just like oh can you review this cuz I know you were really did a lot with Git. Um, and so sometimes I do just pop back and do a little free work, right, on something that I worked on in the past. Um,

and I still like in in the Debian community. I'm still involved with the Debian community. That was like my first big open source project. I still work a little bit in the Ubuntu community here and there. Um, and I I still have some very good friends from that time. But just making sure that you I mean I wouldn't say that it's like a lifelong commitment to every

open source project you touch, but understanding like feeling that commitment and understanding that that was something that you did and something that you were responsible for and just a gentle handoff would be my my strongest suggestion. Um and for me, I mean I never like I love doing all this stuff. So it was never like work work for me, right? like some of it was work work

but like you know I didn't mind hopping back on OpenStack for a few things here and there even when it wasn't my job anymore just to ease that transition because they they the people I worked with were my friends and colleagues and like I cared about them and I didn't want to leave them just you know without my without my expertise there just because my job changed.

So just kind of be mindful of that. Don't go overboard with it necessarily, but just be thoughtful about the fact that you are a key person to the project and don't just drop it off the floor and disappear. >> Oh, we have another one. >> Thank you. Hi, thanks for the presentation. Um, I was curious about your experience with softwaredefined vehicles. Um, did any of that work

ever take you to transit agencies? like was was there any overlap with that or was it mostly like you know light duty single-use vehicles? >> That's a that's a I it did not tren move move over to other ones because the focus was really like I mean it's that one is more of a working group. So they really are focused on like doing research and like collecting

input from the industry to just like put out into reports and then discuss of what the the like the biggest problems are I guess in in the space. So they really were like hyperfocused really on like individually owned vehicles in that in that case. Yeah. But I think there's there's plenty of opportunity and one of the reasons I put together this talk is there's so many industries

we're not in yet that I think we need to be in. Um and the we're kind of like I think we got the lowhanging fruit already and by going into like automotive and stuff we're starting to do harder ones. But I think there's industries out there that we have we really need to bring some more open source into um because there's a lot that can be shared

between We have time for one or two more if anyone in the audience has any questions. I know everyone is um looking forward to lunch. So if not um thank you all for coming. Yeah, give Elizabeth a hand everyone. That was a wonderful talk. Thank you. >> Yeah. And we'll be back here at about 2:30 for another talk. Um and for those of you who came a

little later or u maybe want a refresher, the slides and the uh presentation will be posted on the scale website uh coming soon. All right. Thank you all. Hello everyone. Oh, that does amplify. Okay, great. Um, I'm Amy Fmanian. This is Warpft and Code. Textiles is the hidden foundation of computation. Thank you for joining. Uh, out of curiosity, who all here knits? Yeah. Uh, who all here

weaves? Nice. Uh, and who all has heard of the jakard loom already? Okay, about what I expected. Great. Cool. So, let's get into it. Uh, first I'll introduce myself. Then we'll spend the majority of the time talking about weaving and computing history. Then we'll talk a bit about knitting and crocheting. And then some similarities I see between the fiber arts and open So yes, I'm Amy Fmanian.

I am a software engineer. I've been working for about 12 years in Android development, backend, and recently management. But actually longer than that, I've been attending scale. I first came here with my dad 18 years ago in 2008. Uh after that, we became an Ubuntu household. Um, I had very little idea what was going on at the time, but each time I attend, I understand a little

bit more and I always feel a burst of inspiration from the energy here and all the cool projects everyone's working on. So, I'm hoping to contribute a little bit of that back today. Um, and outside of being a software engineer, I'm also quite the clothing enthusiast generally. I do a lot of sewing mostly and making a lot of my clothes, doing alterations for friends and family. And

recently, I've been reading a lot more about textiles, really digging into textile science and and the history behind all of that. And I had like heard of the Jakard Loom before, but never really like dug into it. So, this talk is the result of some recent digging I've been doing, and I wanted to share with all of you. Um yes. Okay. So let's get into it. Uh

first we will talk about how weaving works and how the first computer was inspired by a loom. So first we're going to start with a bit of terminology to get on the same page here. Woven fabric is fabric that's made by interlacing two or more threads at right angles to one another known as the warp and the weft. The warp are the vertical threads or yarns and

the weft are the horizontal ones. We are going to be saying those words a lot during this talk. So try to remember them. A way that a lot of people do remember them is that warp is like warp speed ahead. That's vertical. Weft sounds like left. So left, right, horizontal. And actually we're already at our first computing term besides thread because that one's obvious. But uh a

warp is a fundamental unit of execution on the NVIDIA GPUs to mean a set of 32 threads operating in parallel. I thought that was fun. That was directly borrowed from weaving more recently. And then a loom. We're going to talk some about some very complicated looms during this, but at its core, a loom is a device that holds the warp threads under tension to facilitate the interweaving

of the weft threads. So here we have a loom at its most basic and the warps are static and the weft there's a lot of flexibility So like that last one um our first loom here is a frame loom. Uh you can't you have the warp threads there. You can do your interweaving of the weft over under over under over under or with whichever pattern you prefer

to do. It's really great for artistic expression. You can be very flexible with the different types of yarns, different types of weaves. You can make nice tapestry, do lots of cool things, but it's not great for speed. If you're trying to weave the sail of a boat on one of these, like it's going to take an impossibly long time. So some very smart people thousands and thousands

of years ago came up with the idea of actually separating groups of warp threads so that as you pass your weft through you have a very simple operation. You just pass it right on through. And this is what's called creating a shed. There are many ways to do this on different types of looms like backstrap looms or warp weighted ones. We're not going to get into those.

Um we're going to talk about these floor treddle looms which is probably the type that you have in your head. Um, yes. So, we create a shed by grouping different warps to these frames called harnesses. As you can see on the chart there, they're also called shafts. Um, with weaving being as old as it is, there's like multiple terms for everything. We're going to call them harnesses

for this talk. So, you thread up these different sets of warps to different harnesses um to separate them. So here another gift to make this a little bit clearer. This person is separating maybe every other warp and just passing that weft right on through. That's a much faster operation than our over under over under over under. And the different ways you organize those warps into harnesses, the

different kinds of weaves you can achieve. So some basic weave structures. I promise we'll get into more computing in a bit, but to learn a little bit more about fabric first. Um, we have a plain weave, which is where you go over, under, over, under, over, under, like we were talking about. Uh, this is very common in many fabrics today. Um, one you might be more familiar

with is like shirting, like for a button-d down shirt, a lot of those are plain weave. And you need at least two harnesses to make one of those. Then we have a twill weave. These are more hardwearing textiles and it's created by passing the weft over one or more warps than under two or more with an offset in each row and it creates this distinctive diagonal pattern.

You're probably most familiar with it with denim. Um there you have your warps dyed in indigo and your WS are white and that's why it appears more blue on the front side, more white on the underside. And that is a twill. And you need at least like three or four harnesses to make a twill because of the different combinations you need to handle. Then we have a

satin. Satin, you're familiar with like this lustrous drapey like shiny fabric. It's created by passing the weft over four or more warps and then under one with an offset in each row. Basically, you want to minimize these interlacing points so that the most light can be reflected from that fabric and make it as shiny as possible. It's also usually woven in a shiny fiber like polyester or

silk to really maximize that shine. If you do it with something like cotton, it's uh referred to as a satine, which you might have heard with bed sheets and stuff, but you need at least five harnesses to achieve a satin. And that's because there are a lot more rules for that offset needed. Here we have like a on graph paper like a chart of a satin weave

where you're spacing out those the black boxes are these interlacing points. Basically, you can't have them form a direct diagonal because then it'll just be a twill and it'll kind of ruin the effect of the satin. So, you need to space it out a lot more. And for those of you that have studied algorithms or are really into chess puzzles, this might feel kind of similar to

the eight queens chess puzzle where you try to place eight queens on a chess board in a way that they can't attack each other. It's not exactly the same. You can see on our example there are some queens that would attack each other on the satin, but it's kind of a similar constraints problem in my mind. And the math involved on actually achieving that maybe we don't

need to go in into fully, but basically you need to find some possible offsets that are relatively prime to each other that sum to the number of harnesses. We don't need to remember that. That's okay. But um the idea here that I saw as like a theory by some historians, it's not exactly proven, but um some people have wondered like why the ancient Greek mathematicians even cared

about prime numbers. Like why would you maybe they were just playing around with numbers and trying to see like how everything related to each other. That's very possible. But primes are very important in weaving as we can see here with satins. And also, if you're trying to plan out a pattern, your warps are static, right? If you want to repeat your pattern a certain number of times,

you can't have a prime number of warps. Um, so that's just a fun theory that I thought was interesting. And of course, prime numbers are critical in software today with cryptography and hashing and Okay, back to loom mechanics. Um, as we saw with twills and satins, the more harnesses you have, the greater complexity you can achieve on your fabrics. This diagram just has two harnesses, but um,

here's a modern example with eight harnesses, I think, are in there. And different combinations can achieve different patterns. Like we said, you could because each one would be up or down at any point, you could theor the theoretically have two to the power of the number of harnesses you have in possible combinations. So when we have two harnesses, you've Okay, so it's two to the power of

harnesses minus two because all up or all down are not useful weaving positions. Your W doesn't interlace anywhere. So when we have two harnesses, you have two combinations. up, down, up, down. Um, and it goes up to 32. There's probably looms with many more than 32, but this seems to be in the like standard realm. That would give you four billion combinations. So, here's some fabric examples

on different um numbers of harnesses. You can see they get more and more complicated over time. Or as told in Mario, might look something like this. Um, but we're not quite as detailed as Mario yet because, um, there's a physical limitation here on achieving those combinations, which is the feet of the handw weaver. The harnesses are controlled by these foot pedals called treddles on the loom. And

here in the example, this person has two harnesses, two foot pedals. Great. Um, in this other example, we have 10. I wasn't really seeing a lot of examples of more than 10. Maybe maybe there are. I'm not sure. Um, but if you have four billion possible combinations and then you can only use 10 in a fabric, that really limits you. And you could theoretically use both feet

on two different treddles. That doesn't seem to be a very common method of of weaving. Though handw weavers, you know, you can do a lot of custom stuff. Uh, it doesn't have to be all set up by the loom. When we're thinking of speed and we want a lot more than 10 though. And royalty and aristocracy definitely want more than 10. They want fancier and fancier fabrics.

So, in order to achieve something like this brocade here where you have these like really elaborate patterns, um the um many smart people thousands of years ago invented the draw loom, which today is called a dobby loom. kind of a um we'll get into it in a minute. Uh also called a dbiloom. This was invented in 500 BC in China or maybe earlier and it didn't survive

but that's the earliest record of it. And here the feet are replaced by another person called a draw boy because it was often a child. And this is a massive machine, right? There's like two people working on this. There's someone like up on top and that person is reading the pattern and picking up the different harnesses or I think possibly individual warps as well to make these

really elaborate As you can imagine, this is also like really tedious, boring and exacting work. So that's like really errorprone generally. So that's that's really difficult. Having two people working on every fabric, that's really difficult. Um, and just the TDM of every single row has to be, you know, read and operated on. They could weave about an inch of fabric a day. So that's that's better probably

than than handw weaving all of this, but still quite slow. And uh the like I said, these are called dobby looms today. That's a corruption of the word drawboy. So that's where that comes from. And the draw boy in modern ones, it's computerized, of course. But so here we're weaving about an inch a day. That's not sufficient. Fancy people want fancy fabrics. So um when the draw

loom gets to France in the 1600s, I think uh the government wants to encourage further optimization of this machine. So we have a few different inventors that are on the case. First we have Bazil Bushon in 1725. He develops the first mechanism to control a loom with a roll of perforated paper tape allowing for more automated but still kind of limited selection of needles and an operator

is still controlling it but it definitely improves on the error rate of these machines. Then his assistant uh Jean Batis Falcone improves on this design by replacing the paper with uh sturdy punched cards like those there. Maybe that's ringing some bells for some people. Um then another op another inventor genius inventor seems to work on many different kinds of uh automata at the time like a a

flute player and um I'm forgetting some other things that he did but he was just like a general tinkerer I think and not particularly obsessed with weaving but he got on the case as well and he incorporates some of these elements to um automate the draw boy. So he still takes the paper tape from Bushon and um achieves like an automated operation to roll that paper and

get to the next row and get all the pieces together. Uh he is of course attacked in the streets for this and he's kind of like well I don't I don't really care about this anyway and he kind of runs away and works on like a mechanical duck for some um because he was going to destroy many jobs right he's like having the um amount of labor

needed for each fabric so then many decades later uh Joseph Marie Jakard I think he was actually a draw boy as a child so he's been in in the weaving industry for a long time. He brings everything together many decades later and brings the punch cards from Falcon and the automation from Vocassan and he builds the jakard attachment to actually control every single warp thread individually. So

now we're less concerned with the harnesses and we can control each one like pixels really. And I wanted to talk about these other inventors here because Jakard kind of just like brought it all together. But I think Bushon probably had the like most important revelation here on actually automating this. But of course the person that actually makes it successful gets the name attached to it. Uh and

here you can see the role of punch paper uh perforated punch cards. And I don't remember which one it is actually, but he also allows it to be this like endless stream of punch cards instead of like what can fit on a cylinder that rotates. So he also makes it quite a bit more flexible in that Here we're zoomed in a little bit more. Here's our punch

cards. There's a flexible number can handle many, many warps. And this is possibly the first instance of like interchangeable machine instructions being separate from the machine itself. So this is our first like instance of software potentially. And his improvements allowed for of course half the labor like we talked about. Um and now the weavers can achieve about 2 feet of fabric per day instead of one inch.

So we're 24 times faster, half the labor. This was incredibly successful. Of course, these were also destroyed by the people that whose jobs were at risk here. And this seems a little bit like not confirmed, but apparently people would attack them with their wooden clogs called sabo, which might be where we get the word sabotage from. I thought that was fun enough to share. Uh here on

the right we have someone actually this is a modern YouTube video that I took a screenshot of but uh someone's working on the machine that actually creates the punch cards. So here they're reading the pattern on this graph paper and making the punch cards down here. I was too nervous to like actually include a video in the middle of the slides that it wouldn't work. But it

has a very uh satisfying like kachchunk chunk kachchunk kachchunk of course. Um yeah, and these fabric designs that you could as you develop, you know, your role of punch cards, these these roles become incredibly valuable. Um you could say it's an early instance of software piracy where rival textile mills are like stealing each other's uh rolls of punch cards to get these like really beautiful fabrics because

of course it was like a lot of upfront work to create all these punch cards. But um yeah, also very interesting. Now, these are some of we're looking a little closer at a jakard loom. These are some of my photos from a trip to France I took a few years ago. I went to a silk museum in Leon. And I wasn't planning on these being presented at

a conference or anything, so they're kind of quick photos. Sorry about that. Um, but I think you can see the operation a little bit more. So, here we have, you know, we're on controlled by the punch cards. A certain number of warps are being lifted. And then the weaver here has many different colors that they're going to wrap around and create their their fabric. Um, what's also

kind of interesting here is that the weaver is actually looking on the underside of the fabric as they work. They kind of have to trust that they know the pattern going through. And here you can see the front side of the fabric. So after it has like kind of gone and rolled around, but that's obviously like pretty late to notice your mistakes. So, still a lot of

slow, high precision work, but um quite a bit Oh, and there's our punch cards. Cool. Very fun. Uh here's a portrait of Jakard in silk. And um yeah, this is woven on a jakard loom at the time. So, it it's really like almost like a bit map of controlling every single pixel to achieve an image. That's like the maximum flexibility a fabric can have. Um, but instead

of resolution, you have like a thread count. This is a blanket I found on Etsy. I'll the link will be in the slides. Uh, here's another example that I thought was really cool. This is like a full silk prayer book. It's 58 pages long. It was also woven at the time. And it's estimated that it took over 200,000 punch cards, possibly 500,000 punch cards to do this.

And somehow it wasn't reproduced all that much. There's like 50 copies of it. Um you'd think you'd want a lot more after that, but that was pretty cool. Okay, so looking back at Jakard, um of course this is a huge revelation for the weaving industry. You can develop a pattern once, produce it many times. you can um disseminate one pattern to many mills at the same time

when there's like a big trend. This is the first time our machine instructions are separate from the machine itself and easy to change. Um I think some more like music boxes and stuff could maybe say this but are harder to change. And what I find most important about this is that it formalized the binary nature of weaving and made it obvious to non-weavers. So of course weaving

was developed over thousands of years but jacquard kind of opens it up to everyone else. So the jakard loom was also quite popular in England at the time and it inspired another pair of um inventors or mathematicians really. So jumping across to England in the 1800s um mathematicians have this problem where they don't have a general purpose calculator. So all calculations are done by hand or there's

some kind of shortcuts where you can use these preprinted uh math tables for quick operations especially for like logarithmic and trigonometric information like that would be a lot of work to calculate you know from scratch every time. So here we have like the logarithms of all these different numbers after the decimal point and yeah this is very errorprone I mean it's like pretty errorprone it's very tedious

to create these tables but are critically important for maritime navigation astronomy everything that um various people are working on. So, mathematician Charles Babage is one of these people. He gets very frustrated and he is quoted as saying, "I wish to god these calculations had been executed by steam." He's like over it. So, he goes to work on what's called the difference engine, which is not a general

purpose calculator, but it basically creates the series of numbers like that last table where you put in a number and it just keeps printing out everything in a series based on how it's set up. Um, sadly, this has never actually completed in his lifetime. Um, some people started it and then I think they got into an argument and then he lost the government funding and it just

never really worked out. These images here are um, a modern produ reproduction of it, original production, because it was never completed anyway. But it was designed with it would have been 25,000 parts um estimated weight of four tons and would have been 8 feet tall to just kind of print out some numbers which is pretty wild. So then he's kind of gets bored of that one and

thinks of a more ambitious machine the analytical engine which he does with his uh fellow mathematician friend Ada Love Lace. This is not only a general purpose calculator but it goes even further than that. and it can like store intermediate values and do operations on those. And this machine is programmable with punch cards just like our jakard loom. Um it also separates the storage and processor um

in order to achieve these calculations which of course was important organization in computers today. And again it was never completed in his lifetime which is very sad. Actually I think it might still not be completed. This might be an estimation. I'm not sure. Um, but it would have actually been the first touring complete machine that we know of. Maybe someone did it and didn't document it. Who

knows? But this is the first known touring complete design. And the punch cards are not an accident. Um, Ada Love Lace calls this out herself. She famously says, "We most we may say most aptly that the analytical engine weaves algebraic patterns just as the jakard loom weaves flowers and leaves." And she really sees the possibility of this machine being like incredibly generic and applied to like different

types of patterns and ideas and symbols and musical notes and really goes really broad with that. So her notes are actually considered the first published computer programs and she gets the name first published computer programmer. You've probably seen her name around in some references around computing around software as well. But it's pretty it just like blows my mind that all this was designed like a century before

the first electronic computers. Um there are some people that kind of wonder what if it had worked. That's the steampunk subg genre of sci-fi writing. Um, this is like an important text in it. I haven't read it, but um, I thought that was a that's like a fun thought exper experiment like what if Victorian England had computers? How would have history changed? And then I like this

little blurb from the back where it was like a box of perforated punch cards are in the hands of a lite daughter. like I don't know that was fun. Okay, so looking to summarize the analytical engine uh this is our first touring complete machine. Uh it utilizes the binary control mechanisms of jacard with the punch cards and of course Ada Love Lace is our first computer programmer.

I think she got in some tiffs with Babage about who it really was. So that's why the published uh qualifier is there. Um now the punch cards possibly inspire yet another inventor um across the pond in the United States um when faced with a problem of the 1890 US census. So the 1880 census took eight years to count and the population grew after that. So it was

estimated it would take even longer than 10 years and possibly roll over into the next census and that wasn't okay. So, um there was kind of a call for improvements for the 1890 census and Herman Hollerith, so people maybe recognize that name already, um applies the punch cards to the data input here. So, this is still different from like programming with the punch cards, but utilizing them

for data input. And with those improvements and some others, um the census took six years to calculate instead of increasing from eight. So, that was a huge success. He turns this punch card system into a company called the tabulating machine company in 1896 which later gets rolled up with a couple other companies to form IBM and is the basis of their punch card systems thereon. Now it's

a little bit unclear if this was truly inspired by weaving. Um he says he saw some like you know train operators punching tickets and that's what set it off. he did have family members that worked in the weaving industry. So, it's possible, but punch cards were starting to be more generally relevant across these different fields. Uh, so yeah, in summary, we have the weavers are developing this

binary system over millennia. Jakard externalizes these system instructions and inspires um programming with punch cards and data input with punch cards. weaving actually makes another appearance in computing history much much later in the 1960s with the Apollo guidance computer for Apollo 11 with a form of storage called core rope memory. Uh magnetic core memory was like very popular at the time and it was used um on

the AGC as well for the random access memory but core rope was much denser and lighter weight and used for the readonly memory and it was definitely readon because uh the core rope process also used magnetic cores but instead of changing the polarity for zero or one a wire would go um through the core to represent a or around the core to represent a zero. And while

magnetic core uses used um one core for each bit, core rope could sto store 192 bits per core. So quite a bit denser um a lot more reliable for space travel as well. And that also necessitated like quite the code freeze. It would be months after you finish coding before actually a team of textile workers in the area handweaves the program into memory which is just super

cool. So the NASA team lovingly referred to it as LOL memory as uh little old lady memory because of all the textile workers working on the case. Uh then much more recently um I'm sure there are many many other references to Jakard and Love Lace and everything but um Google released their project Jakard back in 2015 at Google IO. I was at this Google IO and I

was so pumped seeing it in the keynote and everything cuz my two loves were coming together. Uh they wo um conductive threads into what was actually denim was their first um and only foray into this, but it could be woven on traditional looms. So that allowed it to not be this like super niche um eexiley, but could potentially work in standard textile mills. It it basically turned

the fabric into a touchcreen. and they worked with Levis's and had this cool like jacket you could tap but the the applications were really limited. So they in typical Google fashion closed it down a few years later. Sad. Okay. So now weaving feels more like like an ancestor to computing but knitting and crocheting feel a lot more similar to modern programming languages. so taking a look at

what knitting is. Uh knitting is a process of using long needles to interlink a series of loops made by one continuous thread. There are two stitches, a knit and a pearl. And a pearl is basically the back of a knit stitch as well. A lot this could also be considered a binary system as well. There's only knit or pearl. Knit or pearl. You can have different combinations

of those to form more advanced stitches, but at its core, um, that's what it is. Uh, now out of curiosity, who all here is wearing a knit garment today? Okay, I think a lot more people should be raising their hands. Um, these loops, um, allow for stretch. So, these are originally before the advent of spandex and everything. This is the only way you get stretch. and it's

still like the best kind of mechanical stretch you can get in your fabrics. Um, so and we love stretchy fabrics these days. So if you're wearing a t-shirt or underwear or socks, you are wearing a knit fabric. I just wanted to call that out to kind of illustrate how widespread knit fabrics are. It's not just if you're wearing like a big chunky knit sweater. It's in all

of our clothing. Um, and like weaving, knitting has been fully automated today. There are various machines that operate this. If you were at Kyle's talk a couple years ago, you would have seen the knitting clock that was going around to um map out the year. That was pretty cool. Um and when they first came out, this was also the source of some lite violence. but it still

occurred with government intervention that these machines became more widespread as today. One little machine I wanted to call out. This is a modern um mechanical knitting machine. So, it's not electronic. Um but it's also called a jakard knitting machine and it is controlled with punch cards even today. Um and I thought that was just really fun. So, here you can have like one, my understanding, I haven't

actually used one of these, but my understanding is you have one set of yarns for when there's no punch in the punch card and another one for the punches. So, it can make some fun designs. And you just kind of move the little console back and forth and create your knit. It sounds like a lot of fun. And I kind of want one now. Uh oh, yeah.

Here's our punch cards in in the machine there. And now I've noticed that a lot of engineers like to knit. A lot of people in this room like to knit. Um though we may have self- selected to be here for that, but um a lot of engineers, especially uh women that I meet, uh really like to knit. And I think that's because the patterns feel very similar

to modern programming. Now, this is a knitting pattern, but crochet is very similar. And um as you look at it, it almost kind of feels like looking at some really low-level language. Like these abbreviations look a lot like assembly language. There's um it mimics programming syntax in a lot of ways. So here we have some abbreviations with, you know, row one on the right side. You're going

to knit two, pearl two, repeat from the asterisk four times, knit one, and you're going to do that for a total of 21 stitches. It's a lot like our flow control in modern programming today. So, here, highlighting some of these pieces. You know, you want to work in your stockinet stitch until the piece measures one inch from the turning row. That's kind of like a dowh loop.

Um, we have an if statement in there. Many, many for loops, of course. Um, and then there are these things called markers you can put to kind of keep track of where you're at in the pattern or a place to come back to later. So, in my mind, that kind of operates like a debugging breakpoint or maybe a to-do comment, but lots of similarities as today uh

with programming today. Now, crochet, I just have a couple quick things to say about crochet. Um, I was surprised how modern it is. Um, while the other ones we've talked about are like thousands of years old or knitting might be like many hundreds, uh, crochet is possibly younger than the United States. The first known instance of it is from the 1800s. So, I thought that was really

interesting. Here you have one hook and you're um, creating a series of loops to make these stitches. There are many different stitches you can do. And unlike knitting where you have to like loop in the next loop to close that loop, um each stitch is closed as you go through crochet. So you can really have a lot more variability in the shape that you go for. So

here we have some like lace examples or with our toro here. It's these amugurumi animals that are very popular to make today. There's a lot of flexibility. you can do very quickly. Though, it uses quite a bit more yarn than knitting does. And it's not automated today. There are no machines that crochet, which is really interesting. Though, maybe it's because it uses more yarn, it hasn't been

commercially viable. Who knows? But that also means if you see crocheted items in the stores, either that is a knit made to look like crochet or a human being crocheted that entire garment, which is something to consider when looking at the price. Uh now anecdotally I've also seen that like a lot of mathematicians seem to really like to crochet because you can render really complex shapes really

easily that are harder to do on like paper or cardboard or what have you. So that inspired um the 2009 diagram prize for oddest title of the year. Uh crocheting adventures with hyperbolic planes. That's a really fun Wikipedia page to be on. By the way, there are proofs for achieving similar things in knit, but it's quite a bit more planned and um complicated. With crochet, you can

just kind of like run with it. So, looking back on knit and crochet to close this part out, um the patterns feel quite similar to modern programming to to me, which I think scratches a similar itch in our brains potentially. And I just wanted to highlight how it's much more mathematic mathematically complex than may have been assumed. Okay, so lastly, I want to have some notes about

community. Why am I even talking about this at a Linux conference besides that I was bored and it was fun? Is that I see a lot of similarities between the fiber arts communities and So, first um yes, there's kind of a parallel culture here. Knowledge is generally open and shared. Um there's a lot of open collaboration in both of these groups. um a lot of communitydriven improvement.

Someone posts their code, someone else tests it out, suggests improvements. The same thing happens with patterns. Uh weaving drafts, knitting patterns, crochet patterns. There's a low barrier to entry, but like a high skill ceiling. So there's a lot of like deep craft work you can do on both sides, which I think appeals to similar kinds of people. There's an intrinsic motivation in both. Uh I don't think

anyone's here for the money that you get out of open source or fiber like hand fiber arts. Um and that means people are really passionate about the thing itself. There's visible progress, tangible output that's very satisfying. It's both. You can work on something all weekend and see the result or realize you went in the wrong direction and scrap it all, but at least you can like see

the result of it. And it's um compatible with like distributed asynchronous uh creation. You can work with people on the other side of the planet. You can it's like very introvert friendly. You can work on it just by yourself and then attend knitting circles or hackathons to get a social component too. There's um kind of a similar dynamic there. And I see the strength of both of

these groups coming from a reaction to industry. For software, back in the 50s, software was kind of freely shared among academics. Um the hardware was really the hard part. So people just kind of um shared the same software back and forth. And that continued on until software became quite a bit more valuable and companies started closing it off. Um they used to ship machinery with the source

code. No longer did that. then it was like machine executable code. Um, and yeah, kind of closed it off. Then for a lot of people, this probably works just fine. A lot of people, you don't have to be a programmer to interact with a computer and there's like a lack of necessity on understanding it, which is okay until you really want it goes a little bit too

far and people react. Um, so the open source movement is possibly started with in 1983 with Richard Stallman starting the GNU project and supposedly was inspired by his fight with a broken printer and he really wanted to fix the source code and could not. Very relatable. For fiber arts, it kind of follows a similar path in my mind. Um, you know, for a long time in history,

everyone makes their own textiles, their own clothes. Um, but with the industrial revolution as there are factory textiles, it's like not as necessary and there's like I mean more sweat shops to make the fabric. You don't really have to make it yourself. It's pretty cheap and available. So, you don't need to make them. There's less necessity. You can move on to other things. But then here, maybe

it started in the 70s where there's like kind of a reaction on this has gone too far. Like I don't understand what I'm wearing anymore. I want to make changes to the fabric. this the clothing doesn't fit me right like I want to change this and personalize it and this kind of DIY slow fashion movement takes off and continues today so for both I see it's like

a niche group of passionate people that want to change something about how um the industry is working the for-profit products are working um and both communities use kind of similar tools uh for free and open source software. Of course, we have GitHub. That's a very effective way of sharing your software. Uh for fiber arts, there's a site called Rivalry, which I'm sure many of you are on.

Uh where maybe it's kind of feels like a proto GitHub, but you can post your patterns, many many of them for free. This example is $8. And those that are on Rivalry know this has been on the front page for like forever, so that's why I grabbed that screenshot. Um, this fee is more in like an supporting an indie developer kind of way feeling, but people post

patterns and then other people use those patterns for their own projects, make modifications and um, make it their own. So, here we have some examples where people, this first person made no modifications, just used the pattern. The second person changed out the needle and then this last person made some more structural changes to it which someone else really liked and wanted to use that knitting pattern as

well. It seems to be mostly knitting and crochet on rivalry. It seems like with weaving there is a higher barrier to entry with like needing a space for the loom and everything. If you ask my New York apartment owning friends, that's a big challenge trying to get a loom in a small apartment. But um so there's like fewer people and everyone's on like a different loom. At

least with knitting, you kind of all have the same set of needles. It's it's a lot easier to share. Um but it'd be cool if there was a lot more weaving on Now, lastly, with the similarities between these two communities, I was really surprised there weren't more open source projects from what I could find on the internet. If you know of others, please chime in at the

end. Um but I came across a couple of these. Um, this first one being Ada CAD, of course, named after Ada Love Lace and actually another famous weaver, Ada Deetsz, who we sadly couldn't get into today, but um, this is for algorithmically generating weaving drafts. So, that's pretty fun. Actively maintained, pretty slick. Then I came across this thing called Rooom, which was an open- source jakard loom

for educational purposes. I don't know what state that one's in exactly, but happy to see it there. And then this last one, um, OpenNIT, which was an open- source knitting machine, electronic knitting machine, but it doesn't seem to be maintained anymore, even though there seemed to be a lot of people that enjoyed it. Um, so yeah, wrapping up, I just wanted to point out in this talk

how um these textiles and computers are maybe a lot more similar than you may have thought. Uh, weaving has this shared binary basis with computing. Punch cards have been applicable outside of weaving and to both knitting and crocheting flow very similar to programming today. And this shared maker and DIY ethos is I think what brings people to both sides. So I'm hoping there's more cross-pollination going forward

for how similar these are. Um and lastly, I just wanted to call out a couple books that were very interesting. Of course, I mentioned the first two in the talk. I got a lot of information from the fabric of civilization. There's even more in there about like the beginning of financial systems and all kinds of crazy stuff that textiles have been a part of. Uh, highly high

recommended. Um, also from these other two books, I got some good information, fabric and the golden thread. And then we couldn't I alluded to a lot of the social implications of these machines with the lites and this uh subbo but um this book blood in the machine really goes into a lot more of it and applies like um compares it to big tech today. I really enjoyed

this book um and I think it's important to not see the lites as like anti-technology but more like anti- starving to death. And that's something to consider. And um yeah, here we are at the end. I hope you found this exploration interesting. Um I think the two domains could learn a lot more from each other. Um I personally don't use a lot of social media, but you

can find me on LinkedIn if you'd like to connect and talk further about this. And that's all. Okay. All right. Do we have any >> Awesome talk. Thank you so much. Um, >> you were asking about uh other resources online. And so the weaving counterpart to some of this, there's a site called handwaving.net, which you can kind of think of as like the internet archive, but for

weavers. >> I've definitely I was definitely poking around there, too. Thank you for calling that out. >> Yeah. They have like for those if there's other weavers here don't know about the site. Uh basically they've archived like 70 over 70,000 drafts historical drafts of all kinds and there's a free online draft editor for those of you who know how expensive draft editing software is. Um you can

use that. It's like great resource. >> Thank you. Thank you. >> I don't know. Thank you for the talk. It was amazing. Uh, I don't know if it was just me, but um I couldn't keep up with the part where the punch holes uh make it so that you um are able to separate the the um the well make it repeatable and automated. Um I feel like

I need to >> do some on the the loom itself. >> well, it's not fully automated. We've only automated the draw boy at the top at this point. There's still the hand going through and um interlacing the WS, but basically because you've already you've abstracted out the like interpretation of the pattern into these just like repeatable instructions. You can take this whole roll of punch cards and

put it on another loom and do the same thing rather than needing to manually interpret the pattern every single time. Um, I guess m for making copies of it, you'd have to physically make many copies of it with something like this. >> So machine. >> Do you install the punch hole into the the uh the loom? It's here at the top of the loom. And then, okay,

other people can chime in too if you have a better explanation for this, but I think I got it. Um, as the punch card rolls around, there's a series of Which way do these hooks go? They must go like this. And the punch card comes in. If the needle goes through the hole, it stays in place. If it doesn't go through the hole, it pushes to the

side. And then another bar comes up and lifts all those hooks up that then connect to the warp threads that it picks up. Then it's I was trying to find a really nice diagram for it. It was really difficult. Who has a better explanation? >> nice. I think she might or we can >> Okay, we'll go around. Yes. Yes. Um, and for the recording it was pointed

out these like dobby looms that are computerized today. Um, and I think this is like the main pain point in weaving. I'm not an expert weaver and have only had all the warps set up for me ahead of time. Um, but yes, actually threading through every single one of those warp threads is incredibly time consuming. And >> the setup is the hard part of of >> I

imagine a lot of podcasts are listened to during that process. >> Yeah. I I just wanted to bring up I remember reading um about the French Revolution and there was a lot of industrial espionage involved in loom technology because of its significance in sail cloth basically the the naval armadas um which required a huge amount of of cheap you know cloth. Um towards that end I was

actually going to ask if you could bring up the slide with the book list on it again so I can take a >> Of course. Yeah, that's a huge um thing that any government probably wants is being able to get your ships up and running fast. Definitely. >> Anyone else? >> All right, let's give Amy right. Test test test. How is the >> battery good? I might

bring some over. three out of five. everyone. So we have Monica here who is a communications and engagement specialists at Thunderbird. Um and she'll be giving a talk uh on embracing communications and community and learning to love review. So Monica, take >> Okay. Hello everyone. Uh is the sound good? All right. It sounds good to me. Excellent. So yes. So what inspired this whole talk? And I

like I've said I've been coming to scale for four years now. It feels like forever in a good way. And this is my first talk here. But what kind of inspired this was one of my job responsibilities is that I handle reviews on the Android store. And these were some of the reviews, the answers when people were like, I like this app, but hey, this isn't work.

And these were a handful of some of the things that I got back. And I know that if is anybody does anybody here have their own mobile app? Well, a couple people. So, our answers come from either uh K9 or they come from the underbird for Android. So it doesn't say kind of they don't know that you won't know that it's me now. Our users don't. And

so kind of getting this it's like well they think I'm a developer. That's interesting I'm Monica Monica Ahens Madden. Answer the Thunderbird Android reviews on the Google Play Store. I am a pen and paper enthusist. I am a so-called a I am a SoCal native even though I have lived in Atlanta, Georgia for the last nine years. I am a Linux user since 2018 and I am

a communications and engagement spe specialist which means that if you've ever interacted with the Thunderbird social account on Maston or Blue Sky or LinkedIn or Facebook, you've talked to me. If you've ever left a comment on our blog, you've talked to me. And if you've ever reviewed, who here uses the Thunderbird for Android Yay. Thank you. And who here might give it a try afterwards? Excellent. Thank

you. Extra extra credit edit points. Um, and so I'm responsible for kind of being one of the public faces of the Underbird and then getting that feedback back to our many development and support teams. What I'm not is a native Android user. You might notice I have I have a MacBook. I have an iPhone. We don't have Thunderbird for iOS yet. We're working on it. You can

actually download it on TestFlight. And I am not a developer. I have a humanities background. My um bachelor's degree is in ancient Greek. My master's degree is in maritime studies. And so I am I I don't do code. how does a non-developer end up answering reviews for an Android app? And we'll have to tell you this was not by design marketing and communications took this on when

Thunderbird first acquired K9 which was the older app that Thunderbird for Android was built on top of. K9 had a lower review volume at that time. So at the time it was Jason Evangelo and I who some of you might know who's amazing does Inex for every A1. He was on the team then and it was the two of us and we could manage this. It was

it was an extra task but it could fit within our workflow and this was really because our mobile team was even smaller than it was now. It was C. Ketty who was one of the ma maintainers for K9. And it was pretty much just him. And not only was he developing Thunderbird for Android, but like you know answering bug reports in the reo and sometimes posting in

the port forum. And so we were able to it's like okay you don't have to worry about reviews for your app. Now, the Marcom team aka marketing and communications, we've got this and we'll make sure that we try that we get you feedback on what are the pain points in the app. And so, this arrangement worked until it almost didn't. If you haven't been able to tell,

Inside Out is one of my favorite movies. And answering reviews is a journey of emotions. So it felt like an appropriate artwork for this presentation. So when we first launched Thunderbird for Android, this previous workflow didn't work so well and at the start it was really hard. Jason had left. So the Marcom team pretty and the Marcom team responsible for reviews fell to just me. So in

addition to all those other tasks, now I was the only one handling reviews for K9 still and now Thunderbird for Android, which had a much higher review volume. At the start, I didn't even have an Android phone to test the app. And so I was using an Android emulator on my MacBook. That didn't go well. That didn't go well at all. And so I really wasn't able

to diagnose people's problems if I didn't have the hardware. And like I like I said, I was doing reviews solo instead of part of a team. And what that kind of led to was a horrific and paralyzing perfect storm of perfectionism, imposttor syndrome, and feeling that I was failing at an unfortunate character flaw I have of people pleasing. So it's like, oh wait, I can't do this

perfectly. This is horrible. It's like, why did they give this to me? This is awful. They're like they are going to realize I'm doing a horrible job answering re reviews and they're going to fire me and I'm going to be out of open source forever. And also this feeling that I was just letting people who were sending reviews in who just wanted help that I was let

letting them down. And I don't know if any of you have ever had feelings like that, but they're pretty awful. But thankfully the story doesn't end here. it gets a lot better. Thankfully, I also realized that I had a community who could help me and most of that community came from K9 since thankfully Thunderbird for Android was built on top of an existing h that an existing

app with a vibrant port forum that if I didn't have the answers, it was okay. There was this forum with incredibly helpful volunteers who could help me out and help me find the answers. We also had a growing bug triage team on GitHub. And we also had the involvement of our of our also growing community team who were not only trying to reproduce bugs when they happened

because bugs always happen, but they could find workarounds when those bugs did happen. And this is and some of the work the bug triage team has directly led to macros that I could use that could quickly solve problems for people. We had new volunteers who were answering questions on the Thunderbird for Android section of Ozilla support aka UMO. And in addition to these forums where I could

get help, we also had members of the Thunderbird for Android community support and K9 and the mobile development matrix rooms that live in the Thunderbird community matrix space. And in addition to all of these people, I also had members of the mobile team as I was a member of our mobile team matrix room that I could that thankfully it's like, "Hey guys, it's me again." And that's

how I pretty much start every messages. It's like, "So, we've got this issue in the reviews. Is there anybody who can help me solve this?" So, once I kind of talked myself down, said, "This is going to be okay." and you're going to have people who are going to get you through this. Uh I was do the really next important thing which was to find a new

workflow. One of those key things was starting to attend mobile operations meetings. And so it's which is something that we have been doing pretty much from just around the time that Thunderbird for Android launched. And at this time our mobile team is getting bigger. Our mobile team is I think over five people now which considering we we went from one to over five in just the past

year or two is incredible. But I got to be basically in the room where things happen for the mobile app. And so this way if they're like, "Okay, hey, it's like we've been noticing some crashes. We've been noticing these issues." I don't get caught off offguard when people start mentioning them. It's like, "Okay, I know to be on the lookout for for these." And then I get

to do the task of reporting things to them like, hey, we've noticed an issue with people on K9 who aren't able to access Google, which real which then we realize, oh, we need to do some Casa work on that. And that was helpful because this year I got to be like, hey, remember that thing that happened where K9 just got really weird with Google? Let's make sure

that doesn't happen this year. And it went off without a hitch. And so that kind of gives me a great foundation. I've got this kind of workflow of when I get a re review and it's not a simple answer. I know off the top of my head, it's like, okay, pull up the K9 forum first because we are still maintaining the two apps. Um, besides the cosmetic

differences, they are the exact same under the hood. So for people who are so any answers on the K9 forum also work for Thunderbird for Android 2 for versions of the app after 8.0. If I don't find anything there, I go to our GitHub repo. If I don't get anything there, I go to UMO. If I still can't find anything there, I ask on Atrex. And sometimes

if it's just exhausted, this is when we send people to UMO because as much as you want to troubleshoot something in a review where you have really limited information, sometimes you just need to get them to that place where they can put screenshots, where they can put crash logs, and your community volunteers can help you there. Also, prop, I finally got an Android phone, which was Yes.

which was so which is crazy. It's my work phone that I only use to read email on Wi-Fi and yeah, but this has helped so much. I have all I have all three apps and this has just been incredibly helpful. So, this really took the anxiety and kind of the onwe of the first month or two of supporting Thunderbird for Android, it took it way So why

but I think really the power is not only was in that workflow but being just p being just maybe the public face of a much larger team that people couldn't see. And so why a community model helps is especially we had that deep knowledge from longtime K9 male users and support for volunteers. Like we had this really weird issue that people were reporting in re reviews like,

"Hey, the app's fine, but it is just loading the message list incredibly slowly." And one of the things that one of the volunteers had found is that there was a setting with the contactless images that if the app was acting slow, if you turn that off, it could be better. And it's the people who have been using K9 mail for 15 years even at even as the

codebase changed as we started torn it into Thunderbird for Android. But you all know those people in your communities who've been there from the start who know it in this case far better than me or who know it maybe as well as you if you're a de developer who can just be your godsense. This was really helpful. Like I said, this is the first Android phone that

I've had. This is really helpful when you're new to the app and Android in general, especially when there's problems with the app that are actually caused by your Android settings and having people who are like, "Oh yeah, that's just a quirk with the battery settings." It's like, "Oh my gosh, thank you. I would have never thought to check there." And also one of the and also one

of the things which I think is another great thing about open source is that our users are all over the world and this is especially true for the for the Android app and having this support community has been incredibly helpful here. For example, we would have um Italian users, right? It's like, hey, we have this Italian certified email and my ex um attachments are opening, right? And

we learned that, oh, thanks to our volunteers and team members, oh, the this particular email uses an older version of ESM, which we don't support yet, and that's why it doesn't work. And that enabled me to help these users quickly instead of diving down a rabbit hole for an hour trying to figure out why this didn't work. Also, this gives you insight into issues possibly with major

international email providers that maybe we don't have a test account for yet, but I can get on that matrix channel. It's like, hey, is is anybody on, let's say, a major German or French provider having issues? And I can get that answer from someone who's there because we have somewhere between two and 300 people in that channel. That's much better odds than me. And I can put

that question out there, work on other things, come back a few hours later, and I might have an answer there. Also, a lot of people in our support community are very familiar with edge cases. They are probably power users. And so, for those questions that just are probably going to go over my head a little bit, they are an incredible re resource as I'm as I'm learning

to become a user of the app myself. And like I said, this cuts the research time down for issues and it is almost like ha just having this group of experts that whether it's in a forum post, whether it's in our repose or whether it's me saying, "Hey guys, it's me again." and coming to them with my with the the issue from the re view has been

incredibly helpful and has multiplied the work I've been able to do and honestly I wouldn't be able to do my work without this immunity model. So what's a challenge? It was like okay well this is all great and there's no problems. There's still a couple hurdles that we're working to get this working even better. We are still a small support community especially compared to the Thunderbird desktop.

So one of my goals along with the community team this year is trying to grow that especially on Ozilla's reports. We need to build an iOS support community as test flight reaces gain functionality and because that is going to be helpful once the app is ready for release and then once we have Apple app store reviews that we have to answer in addition to Google Play Store

ones. Another major thing is right now I am the one person answering re reviews. That is an extraordinary bus factor. Also, it means that I take a lot of working kind of vacations where I just do a lot of half days and answer reviews the rest of the at time. I'd like to change that. So thankfully we have a support manager who is working on documentations and

processes and platforms. So other members of the support team that we're all that we're all learning like TB Pro support once that comes online donor care and Android reviews. So that way if somebody gets sick, if somebody has to go out or if somebody just wants to take a long desert vacation, the rest of us are able to cover for them. And unlike um and this is

something that I just talked to my manager about is that right now we're Endesk for sort tickets. That's another little earning curve. But really one of the things I would like to do is to extend kind of seats on end desk to people who want to help so we can have internal notes and really kind of bring people into this process. I think that that would be

incredibly helpful. And we are still working on better and more public-f facing documentation that can not only help Thunderbird for Android users but support volunteers when they answer questions on Ozilla's support when people are having problems. So how can this help your open source project whether you are a mobile app with reviews or not? because this has to be more than a hopefully entertaining story with pictures

from two of my favorite Pixar films and Doctor Who. So your comm's volunteers or in our case our comm's paid staff can be one of the greatest things they can do for your project is to be a connector and a problem solver. And take it from me, your comm staff doesn't need to have all the answers. What they need to be able to do is know where

to start looking for the answers and try to avoid the rabbit holes, which let's face it, we're all kind of pretty good at falling down those. The best people for this role are often the ones who say, "I don't know, but let's find out." You want people who recognize the value of community and see paid and volunteer as part of not as part of one team. And

you also want people who can be good working with not too much information. Uh you have a character limit in when you leave a review. You have a character le uh emit when you answer which for German and French when we translate means we have to be very brief because you like I swear 10 words in English turns into about 20 words in German. Uh and you

want people who are helpful, knowledgeable and yet can be professional too. You will just get you will get people who are very frustrated. I have learned kind of more not swear words but extremely strong words in German than I've ever learned from a course. You will also get people who will do wild things like leave a review for the desktop app and it's like sir this is

aies and you just have to be really nice and be like we're sorry you're having trouble but this is for the Android app. If you go to Sumo, we've got a great community there and they'll and they'll help you. So, you can get frustrated, but you can't act with frustration back to your user. So, you want someone who's not going to do that. Also, the right people

help, but you have to have processes to help them. Having regular sync between members of your comms or support team and developers has been wonderful and my stress level has gone down so much since I started sitting in on the mobile ops meetings. Regular contact with members of your comm's team and volunteers. So that way it's just not like, hey, you're hey, you're the person I I

occasionally ask for help, but you know, we're again part of building that community spirit that you're all one team. It's not just us versus them. Documentation, you want to have documentation. I know we say this over and over, but it's true. that helps people answering reviews and the people supporting the ones and also platforms and processes that make answering reviews easy. You want macros for especially common

issues because that just cuts your time down. tags for better reporting. Uh because you want to reviews are an amazing source of commentary on your app and its pain points and you want to be able to get that to your team. And also those platforms and macros and having a common voice gives consistent answers no matter who's giving them. Uh but most importantly, thank your support volunteers,

especially people helping you who are not getting paid to help with your project. Give them shoutouts. Send them swag or tokens of appreciation. Recognize their skills. These people who are are giving the most precious thing they have in this world and that is their time. Recognize it. Do not take it for granted. And so if you're building a new project like we're building with U Thunderbird for

IOS, don't forget that support volunteers are going to be a vital part of your community and kind of think about how you're going to build that up from the start. Don't be like, okay, I'm going to make this great app, but how are we going to support it? Think about how you're going to do that from the very start. And this right now, this is what we're

using for answering things on the Google Play Store. But this can go beyond Play Store re reviews. We get requests for help everywhere, social media, forums, every news letter that comes into Thunderbird, everything can be a support forum. And so if you have this community that can be helping and and if you can have this person who can kind of be putting order and sustainability into that,

it's going to help your project in the long run. And of course that I you know I said give your community shoutouts. These are just a few of the community members who have helped me. So, Bite Hamster and Tahara, two longtime support contributors on the K9 forum. I actually got to meet Bite Hamster at Fazdom uh two months ago. Uh I was there and we were talking

about A9 and he's like, "Yeah, I'm on the forum." So, I'm like, "Oh, what's your what's your username there?" And he's like, "Bite hamster." I'm like and I just like flailed. I'm like you have helped me so much with your answers. Thank you. Uh and so they have helped with with some of those illutions like I was talking about like disabling contact images if you if the

app is behaving a bit slow. Uh Neil's and Panunistall have helped provide troubleshooting on the Thunderbird for Android Matrix channel which helps me write better replies to re reviews than I could on my own. And Apricot Bucket and Jerry have provided um two work rounds that I used in reviews so much. One was for an IMAP issue that that affected the 11.0 Elise and the other was

a background sync issue caused by battery settings especially on Samsung devices. So there's so many I could think but I really wanted to highlighted the E6 resources if you want to get in touch. Well I work for an email company so you can email me. So I am monica.net net. And if you would like to know any of these great resources and kind of how that they

work or hey if you're a Thunderbird for Android person, get involved with us here. And these slides will all be available on the scale website if you uh want to be able to click on those on those links. They're also most of them are in the get involved section at thunderbird.net. So that's it. Thank you everyone. >> Awesome presentation. Um, are there any questions from the audience

that Monica can answer? Anyone? Yeah. >> Uh, not really a question, but I really liked what you said about um showing like uh thank you for your volunteers recently. I took out some volunteers just for lunch just I mean not something like fancy but you know something uh just to show that we really appreciate all their help. So I super agree with >> that is an awesome

thing to do. People like to be recognized and people really like to be fed. So thank you >> from the audience. Okay, if there are no questions let's give Monica a hand. Thank you. >> Thank you everyone. here. Um, yeah, the talk I think was ending in 15 minutes or so, but you all have an extra 15 minutes to go grab food. The next talk will be

in here at, let's see here, 5:00 pm. So, I'll hope to see you all soon. One, two. So, I'm going to start in a few minutes. Uh, I don't see any AV people in the room, so I don't know. I'll just kind of start in time and go. I But Thanks for the work. I think so. I wired myself up. I turned it on and it seems

to work. >> Are you going to give me minutes at the end or are you >> Oh, yeah. Okay. Wait, I have 50 minutes or 45. Uh hello everyone. welcome to the not last session of the day. I heard there's more sessions. So So we still have energy. That's good. Hopefully you had a good nap after lunch. Uh this talk is about stream processing uh with a

little movie metaphor. Um I'll jump right into it. Uh so the first question is why do we want stream processing? Why do we want to pluck events from thin air and process them as a stream instead of say shoving them in a database and then querying? What's the difference? gosh I've been doing it through about since about 2000, but we had this thing called event driven architecture

in which it's an integration pattern where different systems talk to each other and need to pass data around. And uploading CSVs is well, it's good. It's batchy, it's bulky, it breaks, it happens at 3:00 a.m. And if we have real time stuff, we want it right now. And we want to know what happened right now. So that means that if we write it into a database first,

we may create a hot spot on the latest events when we query when we query. It means that um we may uh create hotspots that impede our uh uh um our right speed. Uh or um you know um it may mean that uh we persist events that we didn't really care to to persist because they don't they're not owned by us. So having different people all listen

and somehow take in and store it in a data store is not always favorable. So reasons can you hear me? Okay. This thing is flappy. I don't know. It's weird. All right. So the big picture of uh and feel free to come here. There's no There's no orcas swimming here. No splash if you need to see it better. Uh so the big picture here is that we

want to process those events. We we have data that is coming in over a something. Let's name it Kafka. Uh I'll name it Kafka nom nomally just because uh it's very popular. There's other ones but let's say we have Kafka. There's some producer something is happening. What is happening? A sensor read the temperature of the patient. Uh the temperature, altitude, um air speed, pressure came from a

sensor on an airplane or a weather balloon, uh whatever it is, events, And then there's a streaming service, I'll call it Kafa for now. uh and uh that streaming service is capturing those things and those events are coming in in some manner of speed and order. Once they hit that streaming service, that becomes the ordering mechanism. Meaning streaming services typically don't reorder the events that arrive at

them. They just say, "Okay, I heard it now. then this is at this point and anything I hear later comes after if anybody's asking me and anything that comes before came before that's that's the order so streaming services have these property of serializing the event stream from that you have a dude or dudet that's a stream processor and they would listen to the stream service and start

taking events the nice thing is that the streaming service can kind of buffer and you don't have to read at the same speed events came in um but not forever maybe depending on the streaming service. So you can kind of take your time taking them. Of course if you fall behind there's going to be a lot to read that you have missed. Uh so you want to

kind of match the speed. So the streaming processor should be light. Its job in life is not to do big thinking. Its job in life is to kind of listen to what's happening, do some transforms on the events coming in, typically aggregate them, maybe route them forward, maybe sync them somewhere else. That's about it. That's the extent of it. So, it's doing some lightweight calculation, asterisks on

the lightweight, I guess. And then once it's calculated, it shoots the sum there in the bottom uh to the consumer of the stream processor. What's a consumer? Well, generically it's anybody who's listening to the stream processor. It could be your dashboard just ephemerally inside, you know, the browser listening and painting, you know, how many sales did we do in the last minute, hour, whatever. It could be

syncing it into a database. It could be shooting it at yet another streaming service and saying, "Ah, I have some stuff for you. How about you listen?" Uh, it could sync it in the database, persisted to files, whatever. So, it's this consumer that can take more time to do its thing. So as we saw this process, we can envision a pipeline whereby the source could be a

Kafka topic, could be a MongoDB event stream, it could be a bunch of things and those are events that arrive at the streaming service and then the stream processor listens to that stream service and then The destination can be anything. How many people here code? Okay. So, how many people are are looking at this and saying, "Well, gosh, I could write this in an afternoon, right? That

doesn't seem too complex." uh and it isn't but uh I've implemented a bunch of these and there's a few characteristics and things you need to take care of in order to do it dutifully reliably without over reporting without under reporting with the ability to recover with the ability to absorb some amount of errors and problems plus you'd have to host it. So I'm going to show a

demo in the end of maybe using if I have enough time to to show you how I do it in MongoDB which has a stream processor that just kind of you you say hey listen to this and then spit it out there and in the middle you just put your query. But um but not having to take care of all of this, not having to do all

of what's in the box here uh is an operational advantage if I don't have to host it, take care of it, twiddle with it, recover it, blah blah blah. I just say, "Hey, I want one of those." That's kind of like one of those I guess it's infrastructure as a service maybe or platform as a service. I don't know. So that's advantageous, but this talk is generic

enough to not focus on one technology. Uh so why not database again? Uh uh in databases are designed to be durable. They really want to save your data. We got to give them the respect they earn for that. But our streaming events may be ephemeral. They may actually be things we don't care to persist or we are not the ones responsible for persisting. So there's an advantage

for us to be able to consume a stream. Even if those clicks are going to persist into S3, I might want to just tap into the stream and listen and do some stuff and just see recent clicks before it hit my main analytics and storage forever and whatever auditing and all of that Databases in general work better with chunky stuff. They're transactional. They're acid compliant. They they

want to persist things in a certain way. In streaming, we don't have quite acid compliance. Um, my advice is not to look at Kafka as a database even though it has K tables because it's not built for asset compliance. Streaming services by their nature are at least once guarantees. Meaning an event that I'm reading I might see once or more. So if I'm counting and I'm saying

how much money did I make? I should be prepared to see the same event, the same somebody purchased for three bucks, maybe twice, and I need to take care of it. The streaming service will happily just shoot shoot the same event again because it's trying to be resilient because it's trying to retry a transmission that it thought maybe failed because you lost the tail of it and

you went back and relisted because reasons. So this item potency is something I need to take care of as a listener. And finally in SQL land I call it uh in the tabular world if I wanted to put it in databases if my event changes I will redefine my schema in the database. Somebody said oh it's not just temperature and altitude it's also GPS coordinates. Oh gosh.

Now I need two more fields. Oh gosh, now I need to go and add two more fields to my database. So um uh streaming services themselves typically do not enforce schem schema. You could build it in if you want, but things like MongoDB, you could just write a document with any properties you want. Next document with different properties. It just takes it for it. It's just an

opaque payload. So that's an advantage where the streaming service can take on events with different schemas pass them along. It's the responsibility of the listener to understand that. So if I tried to pour it into a table, I would first need to prepare the table otherwise the event just won't fit. So why not database? That's why. with that Nuri decided we're going to do event streaming. So

now let's talk event streaming. What is an event? Well, an event is any piece of data. And here I'll take the example that the event is a movie ticket. It's a it's a person coming to see a movie at a certain time. That's going to be my event. Typically, what we do in stream processing, we want to know stuff. We want to know how many people showed

up for the 100 p.m. For the 1 p.m. showing that has three movies, Big, Elf, and Jaws. All monoselabic, easy to pronounce, pretty scary. the output that I expect to see from my stream processor, the ones to listens, is to wait for these things within the window and crunch them together and then spit out a number saying, "Hey, we had so many people or hey, they spent

so much money or whatever it is." So just from that, can anybody point out uh a potential issue or problem? Everybody together at the same time, yell it out. So there's 1 p.m. show times and there are people with a ticket in their pocket for this 1 p.m. The event may arrive with somebody with a ticket in their pocket of 1 p.m. but the person may arrive

at 10:5 1:30. Right? So the question is when I'm computing window, what am I computing it off of? What is the time basis that I'm using to compute the window? Am I looking at the time when Kafka got the event? Am I looking at the time that the event says it happened? Because if I have an IoT device that's occasionally connected, they may say, "Oh, my RTC

clock says it's 1 p.m. ping." But there was network and blah blah blah. And then it retried later and then it pinged at 105. So Kafka would have heard it at 105. So what is the basis of time is one question and good infrastructure lets you pick what you want to calculate based on. So for the time we can pick the time from the time of the

event nomally from the event itself either from the payload or the envelope and we make that decision pretty much at time we write the stuff on what we care and then the time the processor works at is typically wall meaning that the time the processor will say, "Well, it's 100 p.m. by me. It is now time to calculate the 12:00 thing that just ended. That's the end

of the last window." It's not going to look at those things. It's It's going to see them, but it's not going to wait for them because how am I going to know that there's still somebody who's waiting to come to the movie? I don't know that. I don't know if they'll ever come. So I have to decide as a processor, well my window starts here, ends there.

So those decisions are important and it's important for you to understand that because the boss is going to say, hey, you know, there's a dude who emailed me three days ago and he said he was in the lobby and what are you talking about? You're like, dude, I looked at the clock. It was 100 p.m. I gave you the results. So the first thing you we can

do is calculate something called a tumbling window. A tumbling window is what I described. We start a cadence of windows. We start the window on the hour, on the five minutes, on the whatever my wall clock, I'm the processor. And then we finish the window at a prescribed time after an interval. and we open the next window right then and It's simple. It's easy to reason with.

It gives you a nice sequential kind of experience. It also gives you a property of if my boss says, "Hey, it's what is this 220. What's our tally for now?" I'm saying, "I don't know. I haven't calculated it yet." So what can I do? Well, I can shorten the window, but then I get short windows. If he tells me what's at the whole hour, what am I

going to do? So the next idea would be to have a hopping window. Instead of saying I'm calculating just one window when the last one ends, I offset into it and calculate one starting every so often. still a length of an hour. But if I want the lagging hour, now I'm very close to the last one that ended. I can ask what's my last hour here at

2:00. I know what's the lagging hour before 2 and the lagging hour before 4:15 and before 2:30. So I get these 15 minutes interval that my boss can come and ask me every 15 minutes how many people showed up for the movie. Fair. Well, this was simple. I can code that we talked about timing. We talked about people with a ticket in their pocket, but that didn't

make actually the showtime. And you know these people, they show up late and they're like, "What happened? Where's the party?" So, what do we do when my 1pm showtime is here and this is a, you know, the showtime and this dude showed up with an ticket for that time, but they showed up later after the movie actually ended outside the window. What should we do with it?

Any ideas? >> Huh? Throw it away. Yeah. So, ignore it altogether is one option, right? Yes. >> Put it in a dead letter Q. Will we will mention DLQ uh dead letter Q's. Uh the idea of a dead letter Q for those of us who haven't heard it is that these are things that I don't want to throw away because somebody's going to ask me, but I

know I couldn't process it. So, I'm going to hold it somewhere persisted. So if somebody Ezra comes to me and says let's do forensic on it, I can look When you put stuff in data Q, it usually is for manual processing. There are algorithms to go and figure it out but uh you usually have to look at it and reason and while your boss is yeah hassling

you >> flag it in metadata that is late and what include it in the next window and say oh this is a lagger but it belongs to the next window maybe maybe there's a use case for that yeah cool so another idea would be to have an allowed lateness. Meaning that I'm defining my window as 1 to 2 p.m. But I will not calculate it at 2

p.m. I will calculate it at 2:30. That way, letting some laggers still pour in that maybe should have been in the window but aren't. How long should I wait? Probably not long. But this is a very useful thing for IoT when you have intermittent connections that you know shoot things every 10 seconds waiting another 30 seconds another minute another two msil so four minutes for those of

us in the TCP world you know give it something reasonable for packets to retry and if we catch it you know we probably covered 99.999 things of the things that really should have been there and operate on the cadence And if not, yeah, we'll have a dead letter Q or we'll discard it or we'll throw a big error and throw our hands and say that's it. I'm

done with this job. I don't recommend everything I saying. I'm just mentioning possibilities. The next thing is kind of the opposite. I could have an event arriving and that event that comes in arrives in kind of a batch. I have my theater 200 seats. I'm starting to intake people. The first 180 show. Another 20 might show. I'm waiting. I'm waiting. The first five minutes pretty much everybody

showed up. And then like, dude, I don't want to stand at the gate. I want to be at the party. I want to calculate this window. Let's close that puppy and go home. Why still hang until the end of the window if the majority of my events usually happen at the beginning of the window? And I'm just kind of saying, hey, for this movie definition, yeah, it

runs two hours, but who really saw the movie? The people who arrived early. So I can have a idle time and say I'm not going to accept any more events after I a certain time happened after the last event I've seen. So, somebody arrived at 1. 105. 110. I put a lag of 10 minutes. 115 occurs. Nobody arrived. 110 was the last AR uh person. 120 occurs.

Boom. 10 minutes since I've seen the last one. Right. The last popcorn I heard is 20 seconds ago. I'm going to take it off of the fire. So, that's the idea of idle time. Something you want to allow. And what does it really give you? It gives you the benefit of pretty much giving an early report of what happened. And if there is a cost to holding

many many many events, many millions, then that cost is kind of, you know, reduced. I don't have to have so much memory or so much Yeah. So much memory to hold all events in flight until I can calculate So what happens when an event shows after the window is closed? Also after I totally gave any kind of concession for late arrival, what do I do with it

then? I'm staring at the guy with a DLQ answer. Dead letter Q. So what ends up in a dead letter Q? What are reasons why an event would arrive and I wouldn't be able to do anything with it mathematically? Well, it arrived really really late for sure. That's how we got into the slide. But maybe the event is mangled. Maybe I got an event but its payload

is mangled. I can't read it. Can't understand it. Now I don't want you to think that it's like broken bytes and you know bit twiddling. Mangled also may mean that my query didn't fit the schema of the event. I'm looking for a field and it doesn't exist. The value in the field is out of the range that I can process. So malformed is a view of me

as a processor. It could be the event is just fine and I'm the fault. But anyway, I don't want to lose that. I know it has something. I don't want to lose it. I pour it into DLQ. Um, maybe I have validation built in, meaning that I can read it, but it's outside my bounds. Maybe the payload doesn't serialize. It's really mangled. The bites are off. Uh,

maybe the time boundary is violated. I'm getting an event way before a window is ever going to open. I'm looking at the event time rather than the Kafka time. I'm looking at the payload, not the envelope. And the event is like, it doesn't make any sense. Why am I getting the news for tomorrow? It's only today. Uh, and then there could be computational errors. I didn't validate.

Nothing's mangled. I have a divide by zero. I have a something that, you know, happens. And then what can I do? If I halt a calculation, I get nothing. Right? So, I'd rather maybe put the event that caused the math error into a DLQ for inspection and know that because I have things in the DLQ, I should suspect the outputs I got because I can't trust them

because something was missing, right? I have a a big puzzle with one of the pieces missing and I know there's a hole there. I can give you the puzzle and it's like, nice puzzle, but it's not complete. I'm like, yeah, it's not complete. that piece was broken. not everything could be automated. And then this is more of um MongoDB thing. If you're taking out of the change

stream from MongoDB, you could have a scenario where transmits a deletion. it says I deleted something and at the time I'm processing it if I'm saying oh you deleted document one two three well document 3 doesn't exist anymore and I can't read it the data should have been in the event but it isn't so I can't do anything with the last kind of event processing idea is

until now I've spoken of events happening starting at a point and starting butting the last last windows end or staggered but they are all fixed width even in the case I have idle time the width of the event is still the same width the same window I just compute it early and say that's it I can compute and I will wait until you know the next but

you could have scenarios for analytics where you are doing click streams, you're doing things like session tracking and in that case what I want to know is what's the length of a visit of someone. I want to include all of their activity within their visit but I don't know a that they're going to visit my website for an hour or two minutes or two seconds or 12

hours and just be binge watching Netflix, right? I don't know. So for that you could define still streaming analytics but something called a session window and that is defined by having the first event that is associated to that grouping to that session ID so to speak and then some kind of idle time after the last event I've seen from it when I declare it dead and done.

So if I see 20 minutes of inactivity I say okay this session is done. So I define it to be 20 minutes arbitrary number but rather than having a fixed window I have a flexible size window that depends on the data that I see. So that's that one. Uh so yeah so that it closes a window after a gap of time after the latest one and I

define that gap Typically with this by the way the way we do it in web analytics is if we don't see any more events for the session ID and we say okay the session is done and we compute it and we do whatever if suddenly I do get another event afterwards with the same session ID I treat it as a new session because rather than saying hey

it kind of belongs and let me go back in time and yank and call my computation. I say no, I called it a session and the fact that in the human's mind it's it was in the same sitting because they were preoccupied. What can I do? I have no access to that. So in the beginning I showed kind of a and now we're getting into more implementation

detail and the stuff I'm going to talk to you about is a MongoDB implementation detail using stream analytics available inside MongoDB Atlas database as a service where all I have to do is say hey here's where I want you to listen here's where I want you to put the stuff in the end and I'm just going to give you a query to run on everything I in

between. So in order to achieve that effect, I need a way to say where are events coming from and that's a connection. You define a connection and the first stage in your pipeline in your aggregation pipeline in MongoDB will be saying hey use this named connection. I'll configure a connection and I'll say the connection name is blah and what it talks to in this case a database

called stream demo and what collection within this I'm going to listen and then every time somebody writes into that collection this is going to trigger and bring the event into my stream processor listener into and then at the end because I want to do something with all the nice computation I did I'm going to have another connection which says where should it Uh so typically those where

should it go could be another connection. It could be some S3 or something. Uh it could be merge into a different collection in my database. So basically do the aggregation and then store those aggregations. here's the hourly counts or here's the whatever the hourly the patients vitals for the day or whatever. So I can merge it into those uh pre-agregated collections and then if I had that

connection in the beginning and a notion of where to pour it then creating a stream processor is really creating a query in MongoDB where I define a pipeline that says the source is that's my source connection and then the dot dot dot says hey now do something with it do a dollar sum and a a group by an aggregate whatever you want to do a transform take

some fields punch them together I don't know whatever you want to compute so there's different flavors of how we define it on tumbling window uh the stage of computation would be a tumbling window you tell it what interval and what units so two hours and In that window, I want to specify what I'm going to compute. In my case, I want to group by the movie name

and just sum the amount paid. So, I want to collect ticket revenue, assuming there's variable pricing for the tickets. That could be useful. Otherwise, I can just count the tickets and multiply by the price. But this is what I'm doing here. uh for a hopping window I'll define the interval again a unit of hours here could be minutes milliseconds blah but a unit and number of units

and then I'm going to define the hop size the hop size is every how often to start the window window size and how much to hop between the window opens and here I want to maybe do a different aggregation I I want to sum the ticket count and not and not not the revenue. I don't know. It's just examples. If I want to define lateness allow events

that may have arrived after my nominal window end and that's wall time, right? Then I could configure the allowed lateness and say again how many of what unit? I'll allow for 10 minutes lateness. And that lets me capture things that may have straggled it. And if I want to define a dead letter Q, I define it actually not in the pipeline because dead letter Q is not

a T off of the pipeline. It just says, hey, I have something I can't the pipeline can't handle. So it lives outside the pipeline. So to my command to create a stream processor. I'm going to give it my query pipeline and then I'm going to give it options and one of them would be here's my dead letter Q. put all of the events that I couldn't process

into the database my DB into the collection events for review and uh the connection is my DQ connection is because I can do the DLQ right there in MongoDB I could push it to another stream if I wanted so I I give it a all right so I I'm going to show you a little code Well, maybe I'll stop now for questions and then I can show

code and maybe operate it and MongoDB stuff or something. So, let me first stop talking. Any questions so far? Yes. >> yeah. So, the stream processor itself has the notion of a wall clock. It lives in a time zone UTC relative. I don't care. If you tell it, hey, I want to process every hour, it will emit the first window at the first boundary of that round

time in MongoDB. So whether it's 100 p.m. because it's 100 p.m. here or in London, doesn't matter. It's just going to look at the boundary of that time and say, I'm now processing a window. I'm now processing a window. That's all it's really doing. And then the next window will start from there and then offset from there. So whether the event internally came from Timbuktu or from

Brazil or from Florida Yeah. But in terms of considering what is the basis, it will look at the time field in those things and since they are transmitted in essentially bare format, they will be I think considered UTC. The difference is that allowed lateness the if if it's 1 to 1:30, it's not 1 to 1:30, it's a half hour with half hour lateness, right? half hour with

half hour lateness will compute all events that by envelope time or more likely event time. So if I'm using the event time as a reference, I'm saying hey all events that arrive to me within that hour I'm going to consider for that half hour. If they fit in that window I will compute them. If not they will go to the next window. So they will produce very

different computations. One of them "I'm still doing half an hour, but I'm kind of waiting and I'm not holding on and finishing that half hour until 20 minutes later. Ah, there's still a straggler. I'm going to put it here." A window of an hour is going to say, "I'm looking at the whole hour and I'm waiting none." Remember, it's it keeps on moving. So I keep getting

streams of events. I just have to say, hey, am I shoving this event into the next window or into this window? That's Yeah. All right. So, uh yeah, let let me um maybe show some code if the gods of demo work. and this is on uh on GitHub. Um, I can throw the thing up later. So, I have three I have three uh shell windows here. Um,

I have this one file definition which actually defines my aggregation. It has a source stream. Uh is this large enough to see? Should I is it visible there? Yeah. Yes. No. so I I am defining a source stream. It's just a JavaScript, you know, definition saying there's a source. The connection name is by event incoming. Wait a minute. This is all code. What does it really relate

to? Ah yeah, I didn't show you that. So for a stream processor I Oops, sorry. that's not it. I have a connection registry and the connection registry is the place where I get to declare on my platform because I don't want to write all of this on my own. I get to declare what are the available connections. So I created one that is called by event incoming

and by event outgoing and these things. So this is just a name and then processing code in my query I say ah yeah yeah yeah use that connection. What is this pointing to? Well you know I can edit it and decide if it points to a atlas database or to a Kafka or if you just want to practice they also have a practice one. You can connect

to a sample stream for practice to Kafka to HTTPS to AWS lambdas things like that. So you can create connections to a bunch of different infrastructures. So that is where connection name is coming from. Otherwise, you know, I'm telling you, hey, this dude has a DB named demo, a collection named by, and I want to configure because this is a source stream. I want you when you

get the event, give me the full document with the event, not just the ID of the And then I'm saying which time stamp I want to go by and I want to go by the field time stamp. It's a field I have on the document named time stamp. So that's remember how I said you can go from the envelope or the time stamp event of or or

a field in the event. So this how you and then my calculation is just an aggregation pipeline. It's an array of operators. here I'm doing a tumbling window as my first one. It always has to start with a tumbling window or a what's the other one I said there's tumbling and there's hopping. So either tumbling or hopping. I'm doing a tumbling window. So it's just fixed size

start one after the other end. Interval I'm defining 10 seconds. And a pipeline is the actual math calculation. I'm doing inventing a new field total and the value of the field will be a sum of the full document I'm getting and the price and the ID. I'm going to emit a little calculation for each different item skew this stock keeping unit. So that thing takes in events

and for every skew it's going to compute the sum of the price the total of the price that I saw within that time. So how much money did I make off a product? That is what this will answer also. Why not? I'm also going to count. So I'm going to make it easy for somebody to understand. Ah we made so much money by ma by selling so

many and then the final output. I did a calculation where should I put it? Throw it to the trash. Where should I put it? So in my case I want to sorry final output. What did I do? Did I change anything? No. Okay. Yeah. So my final output here that I'm calling it final output. Um I'm just going to merge it again into just because it's easier

for me to show it to you in demo, but I could throw it to S3. I could throw it back to a Kafka stream. Could do a bunch of things. Uh and I'm merging it into a collection in a certain database um under a certain connection name. So then I create a stream processor. The stream processor is just sp.create stream processor. I give it a name because

I could have many stream processors. And then I give it the the pipeline of the calculation which is this array the final output. So my pipeline looks like read from here do calculation pour Uh, ignore this one. This is utterly unused right now. And once I have that pipeline definition, I just create an instance. I start it. And here I'm also showing the stats, but that's that.

So without further ado, I'm going to create that stream processor. It's going to run. And once I have my stream processor created, you'll see in your dashboard, hey, one was created in the cloud. And it's actually in the status of started. So, it's just sitting there. It's going to start charging you money. Be careful uh I don't know 30 bucks a month or something but you know

for the smallest but it's there. It's infrastructure. It just sits there until events start getting fired. Uh I'm going to go into another shell. And in here I have a Oops, wrong window. Thank you. I'm going to uh start a pump. Let me see. And my pump is just a little JavaScript program that I have to induce events going into the collection that it's listening to. Um,

pump events is a very sophisticated file. It has a client. Uh, it creates fake documents and then running it just means that uh, it connects and then forever it creates a fake document and inserts it and then sleeps for a little because demo. So, I'm going to go here. I'm going to run the pump and it's going to start producing documents into the um the database and

the stream processor is listening for changes in the database which is a stream and it just listens to it and can process it. And then finally I need to listen on the other end of the event. And the way I'm doing that here is I have a little listener thingabob. And that listener thingabob is another programming here. whoops. And that one runs a different script that looks

at the collection and every time it sees an aggregated event saying, "Oh, we sold so many things. Oh, we sold." Then it just spits it on screen. So I'm going to minimize this. So the watcher is going to see that things happen, things get pumped in and it says, "Oh, hey, I wrote another document for a window that started here and ended here with a sum total

of a number for a certain school. And then after a little while because it's random the windows are in cadence but it gets produced in random you know a different school nine school four I have numbers from one to whatever and that's all right so this is kind of how it works in I didn't have to spin up Kubernetes I didn't have to write and compile anything

I just wrote a little pipline that represents a query. I register a connection to say where do you want to listen to I wrote another connection that says where do you want to put things after you're done and I am just responsible the query and to spin it all up that's it I'm go my computer is not involved here I can close my laptop go home this

they're happy to charge me money as long as it runs any questions on that or on anything I think I'm kind kind of done showing stuff. So yes, the query. The query that aggregates the query that aggregates the events is ephemeral. It it lives in memory of a certain agent they have up that I don't run. I cannot touch. I cannot do anything with it. I can

only say start processing for me and they will schedule it to work. And that's that. the sync event, the one that I am writing into, since I'm writing into when it says dollar merge, it actually looks up the initial document that's there with that same key, and if it exists, it does the dollar merge. There's also a slam, a dollar out. There's other operators in where you

can just like forget it, overwrite. I'm using the gentler dollar merge, which is probably what you want to use anyway. So there's some read and write but that's not the aggregation that's not the aggregation pipeline that's the output because it's a database it needs to write something >> yeah there's no session scanning Because remember this whole thing is working off Kafka or an event stream from which

is really a connection that you you don't like read it again and again. It's a it's a it's activates you. So you listen and periodically things will come. You don't like go and scan tables all day long. So so that's event processing. That's the reason why it was invented. We don't want to scan tables all day long. We don't want to create hot spots on tables. So

the stream processor merely listens to the stream. It has a scroll point. It has a recovery point. Every so often it writes the last event ID it's seen. And the streaming service typically has a way for you to reconnect and say, "Oh, sorry. I wasn't listening. Can you give me things since that last checkpoint?" But that checkpoint is periodic. Not not every write every event, right? Because

you could have billions of events coming in and we don't want to write to the disk billions of times. >> Another question. Yes. >> Hold on one second. Let me get your question on mic. >> Awesome. well like you mentioned that a consumer it's a consumer's responsibility to ensure item potency processing of the the payload of that message >> have you ever encountered a situation where you've

had to guarantee item potency on the consumer side but you didn't have the pro like you didn't have the resources to maintain the process messages on that side like such is >> you mean that I ran out of memory in order to do it didn't have resource or there was nothing to hook onto to latch on to in order to provide that item >> like the the

obvious way you could do like the really easy way is you attach a UYU ID to the payload and then you save the process messages on the consumer and you just don't do anything if the ID is already in your table for example and you just clear that table X amount of like really simple >> have you ever encountered a scenario >> you've had to guarantee item

potency but you couldn't maintain the state of processed messages within a certain time >> Um, I haven't we had challenges where that presumed UYU ID didn't come from Kafka. It wasn't Kafka. It was a platform that rhymes with snailforce. But, uh, at the time they didn't have that on the API we were exposed to. And we were challenged with knowing whether we've seen that contact before or

not. And we ended up looking at the values and hashing them and just deciding that we've seen it if it's exactly identical valuewise, which is dangerous. It's doesn't necessarily mean it's a new one. It just means we've seen that value before. Uh so that was kind of a challenge situation. Uh other than that, like you said, if the event source has an ID, you can carry it

through. It's annoying because it pollutes your target, but you know, what are you going to do? You want correctness, you pay the price. Yeah. >> So for for most cases, uh it would be a simple simple and reasonable solution would be to append. Well, for most cases, if my stream is known and I'm listening to a topic, I have the envelope ID and that's trustable because it's

that that I listened to before. So, if Kafka is going to give it to me again, they're going to give it to me again with the same envelope ID. So, that's very reliable. >> Okay. Thank you very much. >> Yep. >> Okay. Well, uh, I could make more puns about events, but this event has concluded. That window has closed. I'm happy to have late arrival and entertain

other questions, but I want to let people go on with their day. So, thank you very much for coming and asking and engaging. Thank you. >> All right, show him some love. Thank you >> Thank you for coming out to scale. Enjoy the rest of your day. for time from you should test. Yep. Looks like it's okay. test. Okay, perfect. >> Yes, I agree. And then with

my luck, I would make that Okay. So, we'll go ahead and get started. Um, today's talk is on brand building and open source and why it matters. And I understand it's a little strange to have someone in marketing talking about open source. It might seem a little bit like an oxymoron um because marketing is normally associated with things that are very different than open source um but

marketing is very important in open source but in a different way than you might think. So my name is Shannon Harper. I am the marketing director at DBaver. We are an open-source database management app and um we'll go ahead and get started. It's easy in open source to eventually make a mistake. It's easy to start an open-source company and um to reach a point where the question

comes up, how are we going to make money or how are we going to increase the amount of contributions? How are we basically going to monetize this? And this is where mistakes happen what a lot of companies do at this point is they start to look at their users differently. They start to look at their users as an incomplete paid customer and they start to think about

how they can all the different ways that they can convert all of their open-source users into more contributions into more revenue. That's very dangerous and it's really not open source at that point. It's easy to make that mistake though and we'll walk through how to avoid it and the right way to look at open source as it um contributes to an effective marketing and business model. So

when someone um changes their tune on open source, I'm sure we've all seen it happen at many companies, um it really destroys trust. and you can't get that trust back. One of the great things about open source is when you give something away that is useful, you gain a lot of people that absolutely love your organization and you should never take that for granted because it destroys

trust. And there's a question a lot of the times that uh open source companies ask themselves and that is okay we need to start making money. What do we remove from our open-source offering? What they should actually be thinking is what we should add. the wrong way of asking this question is what do we remove from open source to forcefree upgrades? The right question and this is

really important. The right question is what do enterprise companies need? What do they value that individual users don't value? free should never feel like it's locked down. It should never feel like it's important things to help you um with your projects. It should always meet individual needs and there shouldn't be artificial limits. Paid is about security. It's about collaboration with large teams. It's built for business needs

and it's about support outside GitHub. It's about enterprise needs. how is this marketing model in open source different than traditional marketing? And in this case, it's all about the user experience. If you look at the Beaever for example, DBaever has over 10 million users, a lot of those users don't need to have the enterprise version of DBER. It won't help them and it's not the right fit

for them. So why should we market to them to go into paid? We shouldn't. We should market to them about what's new in the open source version. We should keep them interested in the open source version because these are people that share the word. They love our company and they're it's like gold. We never want to destroy that trust. It's about the user experience for open source

users and it's about the user experience for our paid users too. Why do free users matter? Free users are the best word of mouth. And I'll give you an example. Someone came to our booth today and they said, "Does DBEver work with ODO?" ODO is a CRM. It's fairly new. You're starting to see it pop up more. And no one really knew at our booth. It's never

that question has never popped up. So I looked it up online and found out that our open source users have already written multiple blog posts on how it connects and what you can do with it. That's so valuable. And it's like having hundreds and thousands of marketers that help us get the word out. Also, the more open source users we have, the better feedback we have, the

more feedback we have on what works, what doesn't work, what matters to them. That's really important to make sure that we're always improving. And also the ecosystem is very important. The more people that are using debaver, the more people can contribute and make debaver a better There's a lot of ways that you can lose when you have an open-source model. And actually, when you have a business

that um needs to make revenue, it's really easy to make these mistakes, but you shouldn't because it'll destroy One of the mistakes a lot of companies make is they will make license changes. They will allow you to use all of these things and then all of a sudden you'll get a prompt and it's like oh like it's not available anymore that's a horrible experience. No one wants

to go through that. Um, and if was built for open source, and it shouldn't have been, then it should have never been added to open source to begin with. Um, the feature payw walls is also a big mistake. And that is when you're used to using open- source, everything's free, and then all of a sudden you get a prompt and it's like to finish your project or

to do something extra that any sane person would need to do in order to finish their project, you'll need to pay. And that's like holding someone hostage. It's really horrible. Um the other thing is the aggressive marketing, the upgrade prompts, the um messages everywhere to upgrade. Um that's a really horrible user experience is distracting. Nobody likes it, but everyone does it and it's just not not When

users trust you, they become even stronger than your sales team. The ODO example is one very small example of users that spread the word and help other users more than any sales team could ever do. And there's three pillars in community first marketing. If you listen to this talk and you um don't remember anything else, just remember these three pillars. They really really matter. One is to

give individuals everything that they need. They should have great features. They should have no time limits. They should have no degraded experience. If an individual user needs to have access to something, they should have access to it. And one way that that actually works in action when it comes to marketing is it's important to not only market to paid users, you should also market to open source

users. You should always come out with updates in the open source version. And you should let your open-source users know when there are new things because they matter. just as much as the paid users matter, if not more. So, you should share helpful tips, helpful marketing, but it should be about ways that they can use DBver that helps them accomplish their goals. This is an example of

um our newsletter that goes out to everyone and it includes a lot of updates that are available in the community version and those updates are just as exciting and just as useful. Um they just fit different needs and they're really important. Second thing is focus on helping. It is so much more important than selling. Um, when you build a product that opensource users love, that's more important

than building something that people buy. Um, the fact that they love your product matters more than did they make a purchase or not. And an example of that is um at DBER we have um webinars for our open source version. The webinars are not focused on anything else except for Dave here in the in the audience. Dave will share everything that's new in the updated version. There's

no selling. There's no jargon. It's just all about what's new, how it works, and a walkthrough of the things that matter to our users. Also, never punish open-source users. You should never give them a degraded experience to the point that they resent Rule of thumb, if you wouldn't pay for it as an individual, don't pay wallet. So, what does success look like in marketing for an open-source

company? Success is a little bit different. It's not just revenue. It's about building trust and about building advocacy. At DBaver, we have very strict brand voice guidelines. We are very careful with how we communicate. And it's not because, you know, I'm a type A marketing director and I want everyone to do things a certain way. It's because we are very careful never to use jargon, to always

be direct, to be focused on the user, to not talk about ourselves, to always talk about what it means to our users first and then us. And it's really so important because our users think of us as a friend. And if we do not communicate to our users like we would communicate to a friend, then we're letting them down. And the more people that know about DeBeaver,

regardless of if they pay or not, the more conversations are going to happen. Eventually, someone will be in a meeting and their boss will tell them about how this problem where they acquired a new company and they have a new database and they need to migrate everything over, but no one knows anything about the SQL. What do we do? People will raise their hand and say DB

Beaver can help, you know, and they'll think of us. That is what success looks like when it comes to building advocacy that translates into revenue. We are not selling, we are helping. And when the conversations come up where um more features are needed, then it naturally turns into something different. And um our CEO likes to use this analogy. I think it's a pretty good analogy. Um she

likes to think of open source as a guitar. You know, like you give someone a guitar, they like to play their instrument, they get really, really good. Someday though they might decide they want to buy an electric guitar and they might want to start making money and going on the road. The electric guitar is the paid product. The regular guitar that helps them where they love to

play that is the open and um it's not about taking anything away. It's just about different use There's five lessons in community first marketing. One is open-source users are not failed conversions. We don't look at open-source users as a funnel. We don't look at it as okay, we um you know 10 million open- source users. We have to convert 50% of them and then they go through

and they become opportunities and then we have to close them. We don't look at it like that. We look at our open- source users as like impressions. We look at them as um advocates. It's so much more valuable to have 10 million advocates than it is to have 50% of them convert, not really happy, and then churn. So we always want to look at open-source users as

advocates. Nothing else. Even if they don't convert, we don't care. We care about them being happy and loving our The other thing is to ask the right question about features. When you come out with a feature, it's important to make sure that it's in the right edition. So, we always come out with new features in the Beaver community, but we also have features that matter more to

business users that are maybe more security related. For example, with um our paid product, it's more important that we have like strict AI output settings. That matters to a business. It doesn't matter as much to an individual user. So, it's really important to ask the right questions about features. It'll happen sometimes. We really try to stay away from this ever happening where we come out with a

feature in open source and then we are like, "Oh, no, actually this is more of a business feature." And then turn it off. I think that's happened like once ever since Steve Beaver has existed and we try to stay away from that at all cost because it's not a good experience. Trust compounds over time. I like to think of our open-source users um like a 401k, right?

That 401k you don't expect to take money out and to do anything with it, but it does accumulate extra value over time. And if you have it there, it can turn into other things. And another good example of that is at DeBaver, we offer our paid versions for free with academic licenses. It is such a cool thing to have conversations at events like these with people who

learned how to use debaver in college and then they decide they really like database management and they become a DBA and they start making money doing it. That's really rewarding and it's um a really good example of trust compounding over time. The other thing is the community creates network effects. The ODO story really um emphasizes that and organizations pay for what they value. So if there's a

lot of value in your product then and the enterprise features are valuable to enterprise businesses then they will pay for them no problem. But you have to respect your users. You have to build for them, not against them. And this sounds like such a silly thing to build for them and not against them, but um it's so easy in marketing you know, in marketing, you go to

college, you work for enterprise companies, and they teach you about jargon. They teach you about how to do a marketing campaign. They teach you about all of these things, but when you work for an open-source company, it's really the worst thing that you can possibly you treat your users like their friends and you talk to them in a way that colleges tell you not to, it actually

produces much better results. So, it's really important that you respect your users, that you build for them, not against them, and also that you market for them and not against them. It's better business. Every open-source user could be your best advocate, and open source users are your foundation. They're not your failure. They're not incomplete. They're not something that you convert through a pipeline. Um, they're actually the

whole reason why we exist as a company. And it's important to never forget that. So, um, it's important to think about with everything that you do as a company, if it's a new product that you're coming out with, if it's a new feature, if it's a new marketing campaign that we're building, it's important to think, is this something that's helpful? Is this something that is marketing, And

if it's traditional marketing or if it's not helpful, then it's focused on short-term revenue. And that is not good. It's really important to think about term revenue and long-term brand advocacy. It's important to build products that your users love to recommend. That's the most powerful marketing that you can have is a product that users will share with everyone. Actually, um in the crowd, we have uh someone

in our team. it's her first time going to a conference and um she said it kind of feels like we are like um celebrities because people will come to the booth and they will just say Deep Beaver I love Deep Beaver and like point and it's it's absolutely crazy as a marketer it is like really really cool but it would never ever happen if we did traditional

marketing. It would never ever happen if we did not respect our users. They mean everything to us and we wouldn't exist if we were to change this model. It's So, um that's the presentation and I hope that it was helpful. I know it might seem like marketing is evil and it can be, but when you're in open source, marketing can be the best way to understand your

users and to help them. And it's much more valuable than looking at it as a revenue pipeline. So Dave, do you want to come up here and see? Okay, cool. want to run around the audience for people who want to ask >> Yeah. Does anyone have any any questions? I think we have a question right here. Dave? This is very interesting. >> Great. Thank you. Um so

in in your model you have like um two different groups which are individual users who don't need for example those security features and then presumably large or you know at least enterprise user that need those additional features. What if that's not the case? So what we what if you have an open source product which is like only for individuals and you know a large enterprise would not

necessarily want that um like you know music streaming or something like that. How would you approach that? >> Yeah, that's a that's a really good the way I would approach it is not really that different when you think about it. philosophically because the way that it works with our open source users is we kind of have this model in our sales team that we don't actually have

sales reach out unless they raise their hand. So if someone really expresses interest, if they request a demo, if they want more information, then we will like sales can reach out. If someone um uses debaver, we can tell like if they work for a company or not. And that is our trigger like okay, this person works for our company and um maybe they triled our product and

there's like other people in their company that are already using Debaver. Then we have like a trigger that will reach out to them and it'll say like, "Hey, noticed you tried the product. That's great. Hope it works out. If it doesn't, that's fine. We love that you're using Community Edition. Um, but there's other people that are using your product. We're using our product at your company. You

can probably get it approved. The reason why it's actually not that different is because that's our triggers. Our triggers are based off of that activity. But if you're working with individual users, you might notice that individual users are um starting to use uh features that point towards like more of the buying signals where um like maybe they're starting to to do more of an activity where it's

actually helpful for them to know about extra things that they can do and maybe the paid version is a better fit for them. And that's where it's really the same because it's like what we do in a different way, but it's it's about like having triggers and making it about them and what they need based off of their activities and not about like what we like there's

no soap box. Like we're not telling them, you know, hey, everyone needs this because everyone doesn't. Maybe the open source is best for them, but maybe maybe based off of something that you know about them, they could find more value and other things. So, I have some really cool software I want to share, but I'm scared of the LLM's like ripping it off and, you know, users

getting to use the stuff that I've worked on without getting to know some of the history and getting to kind of engage and contribute to other people who are using the software. Um, and I was thinking about this from like pre-market economies. We were doing like debt based economies. And it's like how important is it to your end users their ability to contribute back in order to

like stay engaged in the community because like you've just given them this amazing thing and there's like this like debt transfer where I'm like oh my you know I'm really grateful to debaver. >> And then there's the other side of that which is like how have you like are the people who contribute more likely to continue using the software because there's like that back and forth or

do you have any data on that? I guess the question on like how important is users ability to contribute to their like long-term happiness and use of the >> Yeah. I mean that's really about like that's the corporate way of like makes them sticky, right? Like it's um I totally get that because you want to that's where the brand trust is so important where users absolutely love

your product so much that they could get it somewhere else but they love your company the way that you treat them and they want to stick with you. And the we were actually Dave and I were just talking about the contributions thing earlier today. It really just depends on the type of open source company because there are some open source companies that are like um heavily invested

or owned by larger companies and they really value the contributions. Like to them contributions is like their version of revenue and or it could be like like us like we love contributors and we have like a undercover program where we'll send like swag out to our our best contributors >> we actually have our repositories on GitHub and yesterday we had a guy who saw one of our

trouble train says I want to work on this please please pick me. So that helps, >> but we're not like dependent on it. Like it's not like one of those things where we look at it um like I'll share it sometimes during board meetings as like cool look at this, but it's not our measure of like success. But I think probably before I was with the company,

like when DB Beaver was first starting out, contributors were probably much more important than they are now. But I would say like a contributor is not a signal of how much they love you. >> Well, we also publish guidelines of how you can contribute and how you can do that. We also mark certain bugs good first issue or we need help here. So, don't be afraid to

ask for help. >> Yeah. And I would also measure like um how long they've been using your product, how often they use your product. Um, DB Beaver, we actually don't even track usage. Um, because we're an app, we're on prem. So, unless they're using like our beta, we can't look into that. Um, but like you can see like how long has someone been using your product and

are they contributing? That's a really good way of telling. And then just treat them like they're special and they're like a friend. Keep them up to date and it's a really good way of keeping them. It's a good question. >> Any other good questions? >> Um, I think I know the answer to this, but I'll go ahead and ask it. So, the paid users, you don't go

with traditional marketing to them either. I assume it's the same marketing, or do you shift how you communicate with them? And I have a follow- on question. >> Yeah, the paid users Paid users are the same. It's like we're reaching out to a friend and it's more like trigger based. You know what I mean? Like someone else at their company is doing something. There's a lot of

expansion within the company and it makes more financial sense for them to like um consolidate things. Then we'll work with that as far as our messaging. And then we also treat them the same way as open source users and that we have like DBver Enterprise new release webinars, DBER team edition new release webinars and our goal is to focus on informing and keeping them up to date

and also making sure that they stay with us for a really long time and they also have like priority support. And my follow on question is if if we have a paid application, we also have a free application too, but we're building something that we're considering going open source on that particular module. So it's kind of a backwards >> Um even though we participate in other open

source projects, we have a product that we think would be useful for the market, we want to release it as open source. What would your angle on that be or what would your suggestion be? I >> Yeah, it's a little bit different because then you're actually kind of going a little bit more into the traditional world just to get you like If it were me and I

were looking at it, I would probably look at um if you have any churn or people that have stopped using your product, I would look at them and see if it's a way to get them back and maybe set up like a sponsored messaging campaign instead of a visual ad um to make it more personalized. I would look at um people who are already familiar with your

brand and let them know about it and see how well that works. If that doesn't work, you're kind of um you know, you should go to open source conferences, you should have a booth, you should like really reach out to the community and focus on outreach and building that as much as possible. Are you doing that to um like get more contributors? >> No, we've been getting

it was a need in the industry that that we were approached with and we started just dabbling with it. We said this would be good to have as an open source, but we kind of are in between right now trying to decide what to do with it. >> I really love that. Um, it's really rare, but it would be really cool to have something as open source

where you're not you're you are just looking at it in this way as like brand awareness, making people happy. That's >> By the way, we just turned something from the paid versions into the into the community edition and I was wondering if it was going to cannibalize our our customers, you know, the folks who are paying, but so far there's been no evidence of that. So, Any

other >> any other questions? >> Cool. Well, oh, one more. Okay. Um, yeah, I have a question regarding how you keep the people close, how you keep the conversation about the beaver going. like you said, you don't want to be too aggressive, but you want to be informative. And like we're in a similar situation and people love our software and they're like always happy because it does

what it should and it works perfectly. But sometimes this is difficult for us to get a hold on our paid customers because when we want to check in with them like customer from the customer care side, they're like, "Oh, everything is going just fine." and then it's like slowly fading out. You >> I feel like we're twins because we talked this morning and we have very similar

things. Um what what we try to do is like when we're onboarding it's a long process you know when we're doing the onboarding that's when we like will say like hey would you like free tickets to go to Google Cloud Next for your team? You know, because if you go to these events, you'll get the free tickets. We would like to do a case study. Are you

open to doing a case study? Then they get onboarded. Then you wait three months and then you reach out again and you give them the tickets and you get the the case study. Unfortunately, it's a different world for paid. It is kind of like a carrot thing because you can't they're very busy. They can love you all they want, but they're really busy and it's a really

complex environment. People are leaving companies. There's different things happening. So, um you know, it's just good to provide something of value to them. Um the the other thing is um you know uh always having that uh community manager that can let you know like when someone is happy or like even on our marketing team Dave helps with the community management for Reddit and then if there's problems

but we have um a community manager on our marketing team for like um X and things like that and that really helps us to know like who we can reach out to. Also, we have like NPS surveys that are sent out where it asks how happy they are and if they are happy then that's another way to move forward. I don't really like the carrot thing unless

it's like a very last resort but sometimes you know it does help. >> Yeah. Thank you. >> Any other questions? Cool. Well, thank you everyone. I really love this. I tell everyone it's kind of like a a data party every time we go to scale. So, thank you. We really appreciate it.