Laly Bar IIan, Gilad Shoham Beyond Snippets Optimising Code Generation by Reusing Existing Component

This talk addresses the evolution of code generation, highlighting the transformative role of components and AI in software development. The speaker emphasizes that while traditional code generation methods were rigid and simplistic, modern AI-driven tools—like ChatGPT—have advanced significantly yet still fall short of their potential, being limited to writing snippets rather than developing comprehensive features. They advocate for a paradigm shift where AI is taught to understand components and their relationships, allowing it to operate more like a software architect than a mere code generator. The presentation delves into practical strategies for improving the capabilities of AI in code generation and maintaining a clean, manageable codebase through effective component reuse and intelligent augmentation. Throughout the session, the speakers share insights from their work with the AI system at BIT and demonstrate how to leverage a component graph model for smarter code generation, testing, and integration.

hear me? Okay, you can't hear me. Can't Everything is going so smoothly. It's beautiful. Like, um, let's check the clicker. It probably doesn't work either. It does work. >> Okay, so um, thanks for joining us. Thanks for your patience. Um, I got to say I had I've had two first times this week. One was on Tuesday. I took my first ski lesson ever and the second one

is today. And let's just hope that this goes better than my skiing. Um, so we're here to talk to you about code generation today. And you're looking at this slide. You're probably wondering, okay, what does evolution have to do with using components? and we're here to tell you today that we think that components have everything to do with the evolution of code generation and maybe AI in

general. Our story begins um a few decades ago in the last century uh with some developers who decided uh that they wanted to build the first code generators. Uh so they did that but they did it in a rule-based or templatebased way. So it was wasn't very good. They were very rigid, very naive, and they didn't produce the results that they were meant to. So, um, let's

jump fast forward a few decades. Uh, we have Cloud, we have Chad GPT, we have Gemini. H, and these, um, code generators really made a revolution. Okay, nowadays most of us are using some sort of code generator and um but we want to say that they're still not being used correctly. They're still not living up to their potential. Um and this is what we're going to talk

about today. We're going to show you how to turn them from code monkeys to actual software architects that understand product, that understand features, that understand business functionality. So let me introduce myself. I'm Lai Barilan. I'm chief scientist at BIT. Uh I've been a back-end developer and a data scientist for almost two decades since before AI was cool. And uh in the last two years, I've been focusing

on code generation. So I'm very excited to be here today and show you what we've been able to achieve and what we hope you've you will be able to achieve as well uh using the code generation tool that we're going to show you. Uh this is Gilead Shaw. Pilad, want to come introduce yourself? >> Hi guys, do you hear me? >> Cool. Do you think AI can

solve this problems with display faster than human or slower than human? Who think AI can solve it faster? No one. Okay. So, human are still in control. Cool. So, I'm Gilad Chuam. Um I'm VP of engineering at bit working with LAI in the past eight years. I'm leading the development team and uh apart from this I'm also um an entrepreneur and uh a mentor for startup for

developers for team leaders and an angel investor. Uh I even saw someone in the crowd that I've put some money uh hopefully is doing their job. Um and uh also I have some uh smartarthome uh brand that I have a podcast, a blog, a YouTube channel about smart home. Uh but eventually like most of you what I'm doing is writing code most of my time or in

other words I'm transforming coffee into code like all of you. But uh for me the story is a bit different. Um in morning I'm transforming coffee into code and in the evening I'm transforming code into coffee. Let's see a quick video. Hopefully the sound will work. Let me know if it works because I cannot hear. This is how it look like. Okay, Google, it's coffee time. Okay,

cool. So, this is uh a real demo from uh my home uh making coffee so I can write more code to make more coffee. Uh but back to our story. So why does all API all AI code generator are still code monkeys? We are um we are stuck at this uh stage and in this presentation we're going to talk about why we are stuck here and what

it's required in order to go to the next uh phase in the evolution of AI. So AI can write today code actually sometimes a lot of code sometimes useful code. I'm sure all of you tried it at least once and so many code but probably all of you feel that it's not yet able to somehow even close to replace human developers. So what is holding us behind

and make us stuck here? The first problem is that we use AI for the wrong task. We teach AI to read code and ask them to write code. This is kind of similar to, you know, teach someone to read a book and then ask him to write a bestseller. It doesn't work like that, right? We need to teach AI different task. We need to to teach AI

to play with Lego and not like just any piece of Lego. We need to teach AI to play with your Lego bricks to see how your components, how your code is built, what are your um components and how they connect to each other. So AI can not only write code but connect components to build some great stuff to build features to build product and not to stay

on the uh on the code only uh phase. Um, how many of you uses AI to write some snippets or to autocomplete function? Raise your hands if you use AI for this. Good. I see most of you. How many of you are trying to build a feature in your company code, not as a toy project, in your company code to build a real feature completely with AI?

Okay, much less uh people. AI is good at writing snippets. It's like kind of a power autocomplete which is don't go don't don't get me wrong this is great okay it's boost productivity it improve our delivery it's uh it's it improve how fast we can deliver uh code and features but what we really want is to build feature not to write lines of code right we don't

get paid by writing lines of code but by building products the second problem is how does AI knows about your standard, your tool, your libraries. Any organization put a lot of effort by creating like um um a system that work for his codebase like which tool we use, which tool, which tools we avoid, how we name thing, where we put things. Um a lot of stuff we

are working very hard to make sure our codebase is growing in a good and healthy way and not become like a tons of garbage. But AI don't know any of this. AI just start from scratch any time. Don't know any anything about our libraries and tools. And yes, there are some solution to it. We can give it some context. We can give it our uh guidelines or

stuff like this. But this process is really hard. It's manual process. It's hard process. We need to maintain this process because we keep changing how we do how we do things in our company. So every time we need to update the AI context. So what we really want, we really want the AI to be aware of this inherently. We want to give it automatically to the AI

so we can build stuff that fits our standards. The next thing is that AI duplicates code. Who here uh raise your hand if you ask for AI for something, look at the code at the output and thought to yourself, hey, I know this code. I see I saw something similar maybe yesterday, maybe two days ago. Have you seen this? Yeah, of course you did. Because AI start

from scratch every time. So you know and now you you ask him for something today, he creates a button. You ask him for something tomorrow, he create another button. Another day, another button. Do it 50 times, you have 50 buttons in your system. No consistency, no standouts, no reuse. If you think about um the best or most productive um developer in your company, any of you know

someone like this? Someone who is first very smart. Okay, in general. Second is really um he really knows the text you are working. He's the best React developer, Noode.js developer, Python developer, doesn't matter. He really know the tech stack, but he also has a very good advantage. He's in the company since day one, since the first lines of code. So when he get a task, he knows

everything. He knows wherever um where he can find any component, what he can reuse, who can he talk with about any kind of feature. He's trying to optimizing maintenance flexibility. He's trying to make sure that the code he's writing today will not break the system tomorrow. is focused on import statement like you see in this place. Uh while AI today is focused on implementation on is just

writing the internal code. It don't know how to reuse it just reinvent again and again the same stuff. We all learn at some point in our career about dry right don't repeat yourself if you don't know it. uh and how important it is to make sure we don't repeat ourselves. So we we we keep our stuff consistent. We fix the bugs in one place and not in

20 places. And how much important it is and we all try to follow it all the time. If you see a function that you write yesterday, it will uh you will feel this pain inside of you that this is not okay. But AI AI today is inherently wet, right? everything twice. Okay, actually not twice. It's again and again, even 50 times. Every time it will start from

scratch writing the same again and again. But when we talk about AI that is wet as opposed to human wet, this is much more uh important because as humans we are uh limited by our capability to write duplicate code because we don't code that fast and also we have this feeling inside of us that we see that something is not okay if we writing the same code

again and again. But AI AI don't care about it. He's writing code lightning fast. You can write thousands thousand lines of code just like this and duplicate everything. If you try to take a system a codebase which is let's say 10,000 lines of code and give 100 developers to use AI freely without any restriction just use AI to develop everything after a month your codebase will become

from 10,000 lines a million lines of code. At this point we are as human losing control of our codebase. we cannot really maintain it anymore because everything is so widely duplicated. So we cannot do anything and uh and now we are only uh develop it by AI and this is something we of course don't want at least not today. I don't know what will happen in the

future but today that's something really bad. So if we take a look on the AI landscape in general today um see this u simple uh chart here and we have in the y-axis we have the codebase size uh it's a small or large codebase large codebase in this context is not like a Google codebase okay it's like a small startup with three developers working let's say for

a year this is large enough for AI to not be able to really build features inside of it and the second the x-axis is is about what is the task we give it like it's a small task like a snippet or it's a big task like building a new feature and AI today are good in either of two uh square in this chart either by doing autocomplete

and small stuff uh in large code bases like GitHub copilot or curser I'm sure you tried these tools and they can be really good at these snippets on large codebase or the other family is if you try to build a new project like a toy project a weekend object a personal website you can ask AI like tools like Lavel and Bolt and and many others probably during

this session another tool will come out um in this hour uh and it's good in like okay I can build a complete website in one prompt two prompts and it works pretty well because we don't really care about maintenance and scaling this but what really interesting is this square on the top right which is um building new features in real code bases because most of our innovation

as humanity most of the meaningful code are written inside organizations that build products and they have large code base they have many developers and they need to build features and products and this is where we want AI to be we want AI to help us on this place which we are not there yet so how we get there that's a great question and I'm inviting Lai to

talk Okay. So before we start digging in, uh let's do a little thought experiment. Okay. Um most of you are using code generation tools, right? So um I'm wondering how many of you would uh let AI just be in charge of your codebase? Like if you had to choose between a human developer that's like a really great software architect who really knows your codebase and the best

AI tool today. Um, who here would rather have the human developer? Okay, that seems like most of you. Uh, and who are the brave souls who would uh relinquish their code to the virtual hands of AI? Hey, you in the back. Great. Um, so I I tend to agree. Gilad tend to tends to agree as well because we think that AI is not living up to its

potential for the reasons Gilad explained and for what I'll show you right now. so the thing is why is it not living up to its potential? First of all, it doesn't see the codebase the way that we see it as humans. Second, it doesn't use the codebase that we use it as humans. and we're going to go over four principles that will help us understand how to

turn AI from code monkey to a real software architect who really understands business and product and features. So, let's start. Um, we're I don't know if they can see me on the camera, so I'm just going to go over here. Um, when we look at at code and when AI looks at code, we seemingly see the same thing. We see lines of code, right? But the thing

is there's a very important difference in the way we see it. When AI looks at code, it sees lists of tokens. To those of you who might not know what to tokens are, tokens in our domain are operators, reserved words, uh variable names, uh class names, function names, etc. Okay, so this is what it sees. It sees the terrain. Okay, when we look at code, we see

much more than that. We see functionality. We see functional boundaries, between features, between products, between components. Okay, and this is how we want to teach AI to see code. We want to teach it to see code like this, okay? To understand what a billing service is or a footer or a header or anything like that. We want to teach it to see functional boundaries and think in

components. Most of you are probably familiar with what components are, right? I mean, most of most of us know it from uh front end. So, we know headers and footers and buttons and shopping carts, right? But at Bid at our company, we see everything So, the back end is also written in components. Any micros service is a component. Any entity which can also be shared with a

front end is a component. Any DB handler is a component. Okay. So everything for us as a component. What does that mean? Um a component is a completely independent software um piece of software. It's runnable and it has everything it needs to be imported and used. First of all, it has its own API. It has its own implementation, its own version control like a mini repository, tests

and documentation. So basically this allows us to just import it and use it and it's readily runnable. Now this is basically the essence of composable software. The idea that instead of writing code from scratch, like Gilad said before, both us humans and AI can put pieces together, put components together like a puzzle. How do we do that? The first principle I want to talk about is composites

are better than atoms. And I think this is not just true for code generation. I think it's true for a lot of domains. How many of you play an instrument here? A musical instrument? Okay. Nice. Nice. Uh, but I'm guessing most of you really like music. Think about it. When we think of music, we don't just see separate notes or chords, right? We think about music. We

perceive it in larger chunks of meaning. We think of chord progressions. We think of musical motifs. We think of um the structure of symphonies or songs. This is how we think that AI should think of components. Think in larger chunks of So in our case, this means that we work with components rather than tokens. Why are components better than tokens? For three reasons. First of all, uh

they have a very distinct business or product Second um they have their own API which allows them to be composed together very easily. And the third thing is that they encapsulate their implementation and this is important because first of all it maintains the privacy of the code the privacy of the IP. So the code itself the implementation doesn't go to the end model. The second thing is

that um it doesn't burden the model with unnecessary information with all the implementation details. Think about it. When you put Lego pieces together, you don't need to know how they look like on the inside. It's enough to see how to just bring them together on the outside. Okay? This is also true for components. Another difference is that we make AI look under the spotlight and that's not

really fair. We as human developers, we have access to all of our codebase, right? If we're looking for a certain functionality, we can look for it in other repositories. We are not limited to our repository. But um as as you said before like most of you are using um probably copilot or c or cursor or tab 9 or um other tools like that and these tools what

they get as context is your local repo. They don't see the other repositories in the codebase. So if you ask them to build you a new functionality they will. But then as Gilad said before another day another button right? we're we're just inflating the codebase this way because the AI is not aware to what's in the other So our second principle is holistic is better than local.

We want to give AI access to our entire codebase because again the idea is not to just write lines of code. We see the codebase as a live organism. Okay. And we want AI to be able to augment it intelligently without duplications. How do we do that? We use the component graph model. In this graph, uh this graph is actually like the political map that I showed

you before because it's actually a map of all the business and product functionality in our organization. Okay. The nodes in the graph are components and the edges um the arrows are dependency relations Um this allows our AI to view our entire codebase in components. and uh this brings us to the next principle. We prefer to give AI an accurate context rather than big context. There are models

today that can take his context I think like a 100red million tokens. This is really great but it's not necessarily the right context. We want to make sure that we provide AI not the entire codebase but what it really needs not just the entire code base but what it really needs and the way we do that let's have a little look under the hood on our architecture

is that we have uh we introduced a rag layer rag is retrieval augmented generation it uses our semantic search engine uh which basically indexes you can see the component search index it indexes only the component component metadata the API the you know the the names of variables etc but not the inside implementation some contextual meta met metadata etc. So you can see that the AI is not

directly connected to the component graph where the implementation is the AI actually has to ask the semantic search engine for specific components that it needs in order to generate new functionality. Um this is a very simplified version of the algorithm we use and I'm going to show you just a little bit a little demo of it in a minute. Uh but what we do is like that.

Uh if a component already exists on the graph then the AI uses it without regenerating it. Um if the component that the user asks for doesn't exist on the graph, the component will build a virtual dependency graph. Which means that if uh for example we wanted to build a shopping cart, then uh it will look for uh components like item or a calculate total or a selector

for shipping method. Okay. So it will look for that any functionality or any component that it finds it uses as a dependency. It imports and uses it as a dependency so as to minimize the lines of codes that it generates from scratch. Let's see it in action. Um so let's say that the user is asking the AI to build a header. What we what the that we

will do is actually create a virtual dependency graph. Okay. So we say okay a header usually uses logo, avatar, menu and search box. Let's what what do we do now? We look for these components. We use a semantic search engine to see if they already exist. So lucky for us, logo already exists. Avatar on the other hand is missing. What we will try to do is maybe

create a virtual dependency graph for that. But avatar is a really simple component. So no dependency graph, no need. So the AI will actually generate avatar. We go on to menu. Menu was found but it needs some modifications in order to be used in the way that the user asked. So um the AI will make these modifications and now we have it ready for use. Now we

have search box. We didn't find search box. We generate a virtual graph. You can see it's a recursive process. We uh we generate that look for button and input. We found both of them. Now we can start going up the tree again or down the tree. It really depends how you look at it. But back to the root node and for each component that we have all

the dependencies, we can generate it. So now we generated search box and now we have logo, avatar, menu, and search box. And we're able to generate header. This way we we really minimized the code that the AI is writing from scratch. But what we got was not just new components or modified components. We got an intelligent augmentation of this entire organism that is our codebase and they

are readily usable which is what we actually wanted in the first place. Um now Gilad will show you what else we can do with that and what magic we can do with this graph. >> Thank you. Okay. So um thanks Lai. So uh Lai talked and explain us how we can teach AI to write components and to reuse and to build the correct components and only generate

the code we need. But once we have a code generated by the AI any AI there is a long process uh from that point until we can really use that code and uh in our codebase. So what he what uh we can do now is to see how we can utilize the fact that the AI now create components individual components in order to shorten this pass from

AI output some code until we can embed this code into our codebase because it's always part of something bigger. So the first thing uh we will start with what we can do in the component level then we'll go up to talk about the graph level because it's not only component it's a graph. So on the component level now we can start write testing right everyone write tests.

I hope everyone write tests but everyone hate to write tests. It's hard it's time consuming. Uh it's hard to understand what are the right test we want to write. But what help write better test is to split your code into small components and and write test for each one of them. It's it helped us as human developer to write better test and it helps AI to write

better test because it can now write test for each one for each component individually. The second thing is about linting. Now AI can do the same. We want our code that AI write to be in the standards of our codebase. So part of our um graph of component uh each component like this has like a metadata. Each node has a metadata which contains stuff like the linting

configuration. So now we can give this linting configuration to a to the AI as part of the context context automatically. So AI can lint each component individually and we build some kind of a mechanism that help write that help run this um CI process or this um uh processes or flows for each component individually in parallel. Um then we can do the same for formatting right we

can make sure AI format each component according to our standards. It's also part of the graph metadata. How we can how we want to format each component. Then we came to standards. Okay. Again, which tool we use, which tool we don't use. AI now can do the same validation for each component individually and make sure it follows our standards, our tools, our libraries. Everything is part of

the graph, part of the context And what really make it magical is that we give the AI the power to do all of this stuff before even giving us as a human this output. Okay, today what a AI flow look like is AI please do something for me. AI write your code. Okay, here is the code. Good luck. You need to integrate it. You need to test.

You need to review it. It's your problem. Not my problem anymore. It's yours. But now that we give the AI the power to run CI and to write tests and run tests and linting and formatting and other two other uh flows for each component individually, AI can now run this really fast because it's happening in parallel for all the components it generated and give us then he

write test and fix the test or fix the code depends where is the bug and then run lint and fix the linting and fix the issues. So when so when AI hand over this code to our to us as human it's already being through five or six um review by the itself. So we as human get on the first time that we get the code we get

code which is much more qualified and much more uh fit to our system. This leads us to the next stuff is about the graph because as we said we don't code we don't write code in a vacuum we write code that needs to be part of something bigger some part of like our code base as a live organism it's evolving it's changes it's combined from different parts

who are connected together you pull some string here and you change some it change something there um so now we can do operations the y can do operation on the graph For example, integration. Today, AI give you code, but you need to integrate it. This is not an easy task. Does the API the AI create for me is good? It fits my system. Can I integrate it

easily? Where should I put the things? Where should I put these files? Where should I put this code? There are hard question, hard uh flow to integrate AI code into our codebase. But since we are now working with a graph and each component has its own API and each edge on this graph contains the metadata about how we connect these two components together. This is something the

AI all sees all the time once you work with this approach. You write the code in a way that it's auto integrated by design. So you basically don't need to integrate. Once you have the code, it's already integrated into your bigger system. Another things that we can do is to measure the impact. Okay, we write this header that Lai shows you. But this header is used in

different places in this place in the CFP page, in the homepage and the agenda page. Now, even before we merge this code by the AI, we can check with this header will affect in a bad way one of these pages. And we run the CFP tests together with the new headers that are upcoming. And we see that the CFP page is now broken. And this means that

our website, the dev world website is broken now as well because it uses the CFP. And we can now see it even before doing anything. And what's nice is that we can give this data back to the AI. So the AI now knows that his new header breaks the CFP page. Now we can take the CFP page into his changes and fix it. Maybe he need to

change the CFP page, maybe he needs to change the header page, but he bring it in and make sure the integration works uh works fine. Then again move it up the tree to check if the dev website is now working well. Another thing we can do is check vulnerabilities. We all try to make sure our code is secure all the time, but it's not an easy task.

Now that we have this graph, we can check for each component. The I can check if there are vulnerabilities or security issues. In this case, we have issue in the menu probably using some old or deprecated package dependency and this affect the header which affect all the pages which affect our website. So now we know that the menu is making our website vulnerable. But this again is

part of the AI metadata and data and context. So now we can tell the AI show him this grave graph of vulnerability and he can fix this vulnerability on the root cause on the root component in this case menu and going up the tree again to make sure we don't have any security issues left. Another thing we can do is now to because we have this graph

we can have analytics and insight on this graph with the AI. We can measure adoption. We can check duplications. Do we have the same component the same functionality in two components or do we have inconsistency like the same component in different version in our graph which means inconsistency we do we have like outofdate components in our graph we can now do many many more stuff with our

uh with our graph and not only for us we can do this together with the AI we can now ask the AI question about our codebase we can ask the AI if I change the towel compon component how many component will be affected the answer of course I hope you know is 42 right um and we can ask him other question like do I have duplicates what

is out of date what need what inconsistencies issues I have in my system and we it's now part of our discussion with the AI it's bring a lot of new capabilities to our relationship between us our codebase and the AI uh this leads to finally get an enterprise ready output from the AI output that is tested that is maintainable that is consistent with our codebase that is

integrated that is reusable by our codebase we basically now instead of inflation we get like a Fibonacci effect tomorrow's AI equals today AI plus yesterday AI the components AI generate today and the component AI generated yesterday can be used by AI tomorrow to generate new features and new components composed from these components. This give us much better and healthy codebase um during the long run. Um the

last step is how okay since we have some delay and I want to do a quick demo we will skip the uh autonomous AI. So I will going back to do the demo from there because the computer is there. We'll do quick demo then I will come back to say few other few last words. So um don't go anywhere please. We'll go there. Sorry for this. Okay,

can you hear me? Raise your hands. Cool. That's a bit weird to do it from behind of you. Um, so this is um, so this is Hope AI and it works in the system in the way that we just explained you. Um, so I asked it in this morning um, to generate a website for our conference. this uh dev world website should have a homepage, agenda page,

call for papers and attendees page. Um and now you can see that the um show us which kind of um base um features it needs to build um its design system. It's the platform itself. I don't see this. Okay, cool. Uh let me enlarge this a lot. It's better now. Let me close this and enlarge it a bit better. Now can you see? Cool. So um so

it it it list the features uh that we want to build and then what it creates it start to generate components. Okay this is just one prompt and you can see it generate 75 components doing this. So it start with building our design system. It start with building our theme um button link heading. Okay. And for each one of these component uh we'll take take a look

in a second. Okay. This is the platform itself. This is like the application that compose all these features. Okay. And you can see like the header for example. And you can see this is new header but it's already using the link that AI created and the logo and the uh theme that we created before. Okay. So we have like business domains uh the platform. We have the

speakers business domain. We have the attendees and for each one of them we generate component that uses other component. If we um this is like a new uh uh a plane organization but if we have components that we created yesterday I can now use them as as well and not don't need to regenerate them and if we want to look if we take a quick look on

some code um we can see that each one of these feature has for example a real backend okay so this is for example they call for paper features and it has a graphql schema okay it's not just a front end. It's it's complete back end. Okay. And it register queries and graphql schema and stuff like this. And we can see for the for example in the attendees

list. I will enlarge this as well. Sorry. Okay. Better I think. Uh let me close this so we can see it a bit better. So we can you can see that um we have that in this list component and this list has also documentation for this specific Okay. I don't know why it's not okay. So, we have documentation for this component and we have test for this

component specifically. Um, and we have example for this components. Let me Okay, let's go back. So, we have test for this specific component. We have uh examples like stories for these components that we can see it render in different state and this true for any of the component we generated. So now we get a very very qualified output that we we can really use. These components are

better than many developers will write as of now. Um so let's say a few more words because we need to go off the stage in a second. So just few last words. Uh um cool. I will go back to the stage. Do you see my screen? Why it's not working? Yes. Cool. I'm going back. >> You can use this time until he gets back on stage to

uh >> Yes. >> scan the QR code. >> So um we are going to release OPAI really soon as beta. So first uh let's do something cool. Take off your phones. Take off your phones even if you already scanned it. Um, and what we are doing now, you will scan the QR code and fill your details and I will do a selfie and it will look like

you are taking a picture of us. So, we will feel better and that would be really great. Take up your phones. One like this. Another one. Cool. So after you scan this another one important thing and we finish another QR >> and come visit us in our booth. >> Yes, come come visit us. We are uh over there on the uh left corner next to the ashen

uh food truck. >> Um this is a feedback form. So you can scan this as well and it will really help us to improve our presentation for the next time. Um and we really appreciate it. Uh hope you enjoy and we are on time which is great. Um so if any of us any of you have any question uh you can catch us here or on the

booth. Hope you enjoy and have a great event. >> Thank you very much.