Beyond vibe coding: A controlled and customisable AI software development process

This talk focuses on the concept of vibe coding and its implications in professional software development. The speaker, Philip from Eclipse Source, discusses various AI-assisted development tools like Copilot and Cloud Code, and their potential to enhance productivity. He explains that while these tools can greatly aid in coding, they are not a replacement for critical software engineering skills. The challenges associated with using AI coding tools in complex systems are emphasized, particularly in terms of task complexity and code maturity. Philip advises on how to effectively utilize AI tools by providing context and maintaining high code quality, ensuring that the benefits of AI are fully realized without compromising the integrity of the software development process.

[music] >> Excellent. Yeah, welcome everyone. Good Good morning. It's great to be back at OCX here again after 1 and 1/2 years without OCX. And what's really crazy is 1 and 1/2 years ago the term vibe coding, cloud code, and codex CLI didn't even exist yet. It's really it feels like 10 years where they existed. So today I would like to take this opportunity to talk about

vibe coding and all of these tools. And in particular where vibe coding may fail for your professional software development, but how you can still gain huge performance and huge productivity gains even if you are working in complex industrial scale products. So just to give you a little bit of context. My name is Philip. I work with Eclipse source. And I want just to briefly mention our AI

story because it maybe helps you to frame our experiences and maybe transfer that to yours. So at Eclipse source what we really focus on is we are building specialized IDEs, domain specific modeling tools, and modeling environments based on various open source technologies. And what these tools have in common is that they are usually pretty long-lived complex systems. So it's not something that you just vibe code on

the side. And at the same time being in this tool market we got in touch with AI pretty soon. It was 2023 when our first customers approached us to ask us about you heard about this AI topic. We would like to have that in our tools as well. This was way before even the mass market was hit with AI IDEs. So we got in touch with that

pretty soon and it got us not only excited for those domain specific tools, but it also got us excited in how can we use that for our daily software development work. And that's basically how we started into it. Back then of course the tools that we used didn't look as fancy as the ones today. They are looking They have been looking like scripts that we had on

our machines where you basically specified some custom prompts and you had to really lay out all the details of what you want to have and usually you needed even to end your prompts with backticks to indicate to the LLM that you really want to get code as a result. And that's how we started. In the meantime, of course, we moved on quite a lot. So, we have

these fancy cool tools like Copilot, Cloud Code, Theia AI, and all of these tools that are, of course, now operating on a completely different level and the all of them, of course, claim that if you use them, you become a 10x software engineer and that's basically their selling point. And obviously, there is some truth to that. So, you can get a huge performance, but it doesn't come

just for free by default, by just using them. There is things you need to do in order to gain those productivity benefits. And um they are so autonomous in the meantime that you can basically just do what many of us now call vibe coding. And just the vibe coding is a heavily overloaded term, of course, very diffuse. So, just to define it for the context of this

talk, I don't want to discuss about terms terminology too much, but just for vibe coding is basically that you specify what you want. So, you define the problem space. You leave out the solution space. So, you don't specify exactly how you want to have it solved. And then you just look at the runtime result without looking at the code. And amazingly, this actually works, at least sometimes

or often or to a certain degree. But maybe you share this experience that I have, it quickly crumbles down in especially when you are working on these long-lived complex systems. And the reason why this Grumbles down is essentially based on three dimensions where your project could be located. So the first dimension is task complexity. So obviously if you're working on a rather simple task like you are

in a to-do list app or messenger app that you are developing, it's not just a technical complexity that is much lower, it's also the requirements complexity that is very low. So the LLM can easily fill all the gaps that you didn't specify in your prompt. If you just to give a random example working in an avionics certification tool, the LLM might have a harder time to fill

the gaps in terms of your requirements. Obviously then it will start to make things up and hallucinate and this is probably not what you want to have. The second dimension is code maturity. LLMs typically work very well in greenfield projects, but they start to not perform very well in in brownfield projects. And the reason for that is also pretty clear. Essentially in a greenfield project there is

no architectural constraint. There is no history of weird decisions that you had to take for specific customers to enable this or that use case and so on. And that's why basically the LLM can do whatever it has seen in training the most. In brownfield of course you quickly run into conflicts in architectural constraints and so on. And the third dimension is the stack, the technology stack. So

obviously in popular technology stacks, popular programming languages the LLM has seen much more training data than in very niche products. The problem for us now is that and I think you share this with me, working in complex long lift systems, you are typically typically not in this area, but you are typically in red emoji area where you have long lift systems where you have history of long

architectural decisions Being a not green field project also means that you have these niche technologies that you may need to work with and so on. And this is not just only my experience, there is also really good studies out there. So, I really recommend watching the talk from Igor about AI developer productivity that exactly basically confirms that by looking at hundreds of thousands of developer commits and

classifying what the AI productivity gain is. So, please check out this video. But of course for us working in those complex systems, what does that mean? Is there no AI benefits or no benefits to gain from AI productivity? That absolutely not. Of course, we cannot just change the project that we are working in, but we shape the task scope, we can specify the right context to basically

aim at pushing the task at hand into the sweet spot of AI. And that usually means injecting the knowledge about the code base, the history, injecting the right tools, making the good judgment and judgment calls and decisions and trade-offs based on the history that you all know about the project but that is not specified in an external factor so on. And of course, we can incrementally evolve

the project into a project that is easier consumable by LLMs by basically applying all of the software engineering practices that we've been using since 50 years in terms of making code easy to understand, modular, avoiding too high coupling and all of these things. So, basically our goal is always to push every task into the sweet spot. And to summarize all of that, what that means is essentially

what we need to do is we need to we need to avoid outsourcing the thinking as a result of using the coding assistance because essentially in the end AI can only amplify your thinking, but it can never replace it. And uh that also means keeping the quality of your code base and of every of your change as high as possible because because the study that I mentioned

before also clearly showed that while there is on average an productivity gain of 30 to 40%, which is amazing if you think about it, you immediately lose half of it essentially because you need to spend more work later on to clean up the slop that you've committed last week. And that's by just avoiding that, you can basically double the productivity gain. So that's I think the key

point. Stay Keep thinking and keep quality bars high. So what are the most important ingredients to basically gain the most benefits out of AI coding? The ingredients are three items. First of all, your abilities as software engineers, and I can overstate this enough. Just because AI coding tools can now write code doesn't mean that they can in any way replace what has always been important in software

engineering. That is defining the system, knowing all the background of the software, knowing the requirements, translating those vague requirements into something that you can implement. This is all still in our hands and we cannot just stop doing The second thing of being efficient in applying AI coding agents is understanding how coding agents actually work because this helps you steer sort of the agent's trajectory in a much

more efficient way than if you just consider it being a black box. And last but not least, it's establishing a struct more structured process in applying AI coding tools and establishing the right culture because in the end it's a learning experience for applying AI coding in in specific team and product. and having a structured process enables you to exchange experiences, exchange new ideas, having a common terminal

terminology, appreciate if someone applies AI coding and fails because that's something where you can learn from for the next time. And that's really something that every team needs to do. In this talk, I will focus on mostly not on your abilities as software engineers because there is nothing that I can add to what you already are. All of the people at the OCX conference are usually top-notch

software engineers, so I won't go into any details about that, but I will really focus on giving an overview of how agents work on the one side, and then I talk about a few steps in this structured process in defining the context and what to do when you receive the first feedback from the agent to help you guide a little bit. But I will stay on a

rather high level. If you are interested in going much more into detail, at 1:30 this afternoon, there will be by Jonas, he's sitting over there, a much more in-depth training with more exercises and more examples where you can even then apply this more in practice. So, let's start with understanding of how agents work. So, obviously we all have used an AI agent, and basically how you need

to think about it is there is form factor of using it is typically user chat. And in the end, you communicate with an LLM, but in the middle there, there is this coding agent, and it feels like one thing, but essentially this is really a very, very thin layer on top of the LLM. That's how you need to think about it because all the AI coding agent

essentially does, of course they add a lot of sugar here and there and maybe a little bit of automation here and there, but all of what they actually do is they introduce tools so that the LLMs can basically decide to call these tools in order to explore your code base on the one hand side, propose changes on the other side, and run some processes on the on

your machine in order to gain some feedback of what is currently the state in terms of like a compiler or a terminal command or a linter and whatnot. And basically that's all the LLM's all the coding agents add, and they put that into the context to the LLM. And so really to optimize the usage of AI coding agents, what we need to worry about most is exactly

this context because that's decisive for what the LLM will do next. And what is in this context? I already mentioned it a little bit. Essentially, it's the system message of the AI coding agent that you're using. Obviously, in some areas that's basically unknown what the system message exactly is. So for instance, in Copilot or in Claude, maybe it has leaked or not, who knows. You can maybe

get an idea of what the system messages is. But in other tools like in C I A I or C I D E, it is even something that you own and you can check out and change. besides the system message, there is essentially the entire conversation that you have, which is your first message, all the replies of the system of the LLM, as well as all the

tool calls, tool results, and so on. So this all ends up in every single request that you send to the LLM. And these tool calls, they pull in much more information than you you may think. So they could load entire files, they could read specific files and put them in very specific position in your prompt like Claude MD files or readmes or whatnot. As well as any

other things that you may have hooked up into your system like MCP servers for I don't know, automating UI testing with Playwright or whatnot. So basically, that all ends up in every request that you send to the LLM. So in order to get the most out of these agents, we need to understand exactly that and need to aim at steering the assistant result by considering every request

that you send to the LLM will contain all of that and we need now to optimize basically the relevancy what is in there in terms of information that you specify whether it's correct and not contradicting, whether it's complete for your task at hand or at least completable so that the agent naturally knows where to look and gain this additional information and we need to make sure that

it has the right trajectory. So it means that the next step is naturally leading in the right direction and that's something that is very very important to consider that this whole chat is basically what you will send in the next step to the LLM. So in order to steer that obviously the most obvious part is the user message. So let's look into the first step of starting

an AI coding session that's defining the context for the first user message. Obviously this is like a combination of task engineering and context engineering. So you basically need within one message specify what you want to do and all the information that is needed in order to decide how to do it and that's basically what you need to put into this first prompt. And this sounds much easier

than it actually is. So what is the perfect prompt? Consider you have like a GitHub issue now received on your open source project or on your product and the GitHub issue says dragging too fast in our graph editor for customer XY makes the lines jump around. So that's the GitHub issue, right? And as a vibe coder I know that's not a good prompt because it doesn't specify

what we should do next. So I will just add fix it because that's how you already tell the agent what to do and I know this graph editor is kind of a beast so it's a little bit complex. So I just to be sure I will add think hard because that will make the agent work much better. And obviously this is just because there is no thinking

that you spend into this prompt. So, there basically is no thinking that the AI agent can amplify for you. And how to do that better is basically finding a prompt that contains all that the agent needs in order to amplify your thinking. And the best way at least for me of how to think what to put in a prompt is by observing my own thinking when I

read this GitHub issue. Because when I read this GitHub issue and I'm just inventing this now because it's a fake GitHub issue. When I read this, I maybe think, "Ah, the graph editor." That's actually not called graph editor. That's just how the user called it. In our my code base, this is called XYZ chillest B server XY Z whatever. And I know that actually the lines that

are defined defining where they go is actually a completely different component because this is some routing algorithm that we use and whatnot and so on. And then maybe this all of this information is really just what comes to my mind when I read this. So, I should really add this into my knowledge context when I think about the next step. The next step is how should I

maybe even think about fixing that or identifying even the root cause. There comes another source of information into my mind and that's the history and the requirements and all of that. For instance, I know that the customer XY is particularly known for having huge diagrams with lots of overlapping edges. And this is of course a huge uh and it's very important information in order to already steer

what we need to look at. So, basically the AI has no idea that this is basically what what we need to focus on. So, this is again something that we need to put there. And then in the next step, thinking about a potential solution for this issue, there is of course multiple different directions we could go. We could maybe just do a fix so that we avoid

an async call or we maybe know, "Ah, in the next month we need to add this or that feature." So, we should prepare for uh some other solution. That's again your thinking that you in some way need to make accessible and externalize for the agent. So, basically, this is the kind uh linear context growth that happens in your brain, and that's I think also what should guide

us in defining the right Obviously, we don't If we just put now this GitHub issue into the agent, we skip all of that, and we basically send the AI agent on based on missing knowledge, uh leave all of the discovery of the relevant context to chance whether it happens to find the right code or not, and uh we all of that do in uh we all we

do all of that in just one context window. And obviously, this will lead to unpredictable quality and most likely not to a good solution. So, what we should do instead is really basically uh invest the the knowledge, the thinking, and the judgment, and the and the design, everything that makes up a good software engineer into this process because now is the time in the AI coding process

where we need to invest that time. Uh that means uh injecting the knowledge of the relevant parts of the code base, maybe think about what additional information sources are not in the code base that the AI agent maybe needs in order to identify the root cause like uh logging or tracing. So, maybe it's worth uh adding logging first, run the scenario once, and then copy uh this

information into the uh prompt to think about what is the actual solution. Uh maybe you have weird libraries in there where there is no training data for the LLM. So, maybe you meet may need uh to make certain parts of it accessible. Uh think about the requirements, and of course, the nature of code changes. Is this something where you think a refactoring is needed, or is this

something where you just think a try and catch is needed? So, all of that intention or uh intuition needs to go in there as well. And I was planning to give you AI coding uh demos, but uh then in the end I felt giving AI coding demos in this uh talk always undermines the point because they should be quick and easy and in the end successful and

that's exactly not what I always want to transfer. And the most important part of the demos is actually that by just doing a little bit of what I just discussed, we receive changes uh not like up this here for a specific issue where we have 316 lines changed. Uh that doesn't solve the problem uh but we receive only 168 changes that really resolve the problem by just

adding three or four sentences. And that's uh if you think about it, I mean this is just a small number. If you think about it, that is huge because it means uh if you would accept uh as a vibe coder the first solution, you would add 150 unnecessary lines that more and more erode your code base over time. So, really invest the time here. Um obviously, you

don't need to basically sit down in an editor and write a huge markdown file yourself for this. What you need to do instead is basically use an incremental process where you obviously can use AI again and that typically means in this define step in the context engineering, you can use the AI and prompt it. Uh you gain some context, you think about which part is relevant, you

review and decide which you extract, and you do that incrementally to build up the context. So, it's linear in context creation. That's the uh process here. And uh essentially in the end, you end up with a task context that you use for the prompting. Uh one way of thinking that at least that's for me very intuitive and nice and I have to give props here for uh

decks. So, this is also another uh recommendation to watch this talk uh of how to depict this process is essentially that you can split this whole uh phases that I we just discussed uh here um into uh different steps where each of those steps is a is a repeated prompting of the AI and in the beginning, we are basically scoping the relevant context. Uh here typically, you

have rather short prompts which are more like queries uh where you want to have answers for specific uh questions. And then there is lots of agent work because it's exploring maybe the code base, it's slice, it's identifying the slices through a code base that are relevant for a specific question. And then the resulting thing is knowledge. And you do that to collect all of the things. Then

there is a human review in between where you identify is everything that I at least know about what is relevant now in there. If yes, then let's go on and we talk about the solution or how we want this to be maybe fixed. Here the prompt is a little bit larger because it already contains all of the facts that you gathered in the first place, but the

agent work is already much less because it's much more constrained on reasoning about is this a good solution or that a good solution. And in the end it will then produce a potential task which is the specification of how to solve it. And this is again something that you can very easily review and iterate on. And in the end you have a rather large prompt which contains

all the context that is needed, all of the solutions that you thought about and deemed relevant. And then the agent actually needs to do much less and can really focus on doing the changes. And the result is the implementation which again you of course need to review. So now as you have the perfect prompt, at least for your particular part, let's go and run it. And typically

of there is two two things that can happen. Either it's perfect and done, then excellent, you can go on to the next task, grab a coffee or whatever you like. But very often it's not perfect yet. And the question is now, what do we do now? The intuitive next step is of course to just go into the same chat window and say I didn't like that, I

didn't like that. But what you're actually doing is not just refining sort of the conversation, but you always need to be aware that you're actually creating a completely new prompt that has the entire history of the previous prompt that that have in your history. And this may include uh parts where the agent actually went wrong. So, it already includes the wrong trajectory. So, the much better approach

is to think about this as a new prompt and identify what needs to be changed uh to get to the good solution and just redo it with a new uh prompt. So, the first one is fine if it's just small things, but if it's off by a large, then you should uh basically redo. And of course, there is much more uh things you can do now is

for instance, you identify Okay, this part worked out very well. Other parts didn't. So, maybe let's uh split this task because obviously, there is two different uh contexts that is relevant. One led to the good solution there, but didn't to a bad solution there. So, let's split this task and uh do two uh separate prompts. Uh maybe there is a a frequent uh um iterative uh summarization

to extract what has been done for the follow-up task and all of that is typically steps that you need to uh think about or adapting the project context. So, just in short, uh when you get the first result, try basically to find architectural blockers first and use them as signals for redoing the prompt in a way so that it leads to the right solution that you want

to see. Then, uh if you did if you did that, uh always think about what is the context that you need for the next step and uh try to basically extract uh that and uh decide on the next step uh for uh with that particular context. And in the end, only do a review if it really pays off because in the age of AI, your reviewing capacity

is really the bottleneck. So, only spend it if you are rather sure that this is a solution that you uh will likely keep. Uh but at the same time, don't skip it because eventually, you're still the owner of the code even though you had a machine write it. Uh so, that's uh obviously clear. Um so, how can we make this project up this whole process more efficient?

Uh first of all, always remember that uh the AI agent is a newbie on every new session. So, whenever you start a new session, it has no idea of what uh of what you did before and uh of maybe your entire knowledge of the project. So, that means we need to give it persistent memory. Uh and that is uh most famously, of course, things like a cloud

and D or project custom instructions in Copilot or a readme file or whatever uh to to basically guide the AI coding agent on how to build, how to lint, and how to test, and all of that. They're super important, of course. But, there is many more things that you maybe should consider. Uh so, for instance, uh give it hints in how to navigate the code structure efficiently.

Uh give it information about recurring practices or patterns that you may have or API knowledge. But, at the same time, avoid making this entry document too large because it will fill up on every request, of course, your context window again. So, a typical thing is here is basically consider this as an index where the AI can enter and then decide based on the task which further information

to read. And last but not least, remember that your source your entire code base is in some way project context too because because it is it is explorable by the agent. So, if you have strange weird patterns in there that you don't want to see in the solutions, you better don't have them in your code base because otherwise they will end up again in your suggestions. Um

and one uh overall um recommendation that I can give you, especially if you introduce AI in new um observe how the agent navigates your code base because from those trajectories you can learn what the agent tried, what it looked at, and what how it used this information for deciding its next tool call. Uh and with that, you can essentially adapt this context that the agent gets in

order to give it in the right direction because every time the agent went in one direction, ended there, and went in another direction again, is essentially collecting wrong irrelevant context in the in the in all of the future sessions that you have. So, you need to cut that short and adapt your code base accordingly. Another thing to make this process more efficiently is of course the task

context. You really need to consider, I think, the task context as your working memory for this particular task. Keep it up to date, have it as an external file that you can read, that you can modify, and where you basically collect all of the facts, the states, and the constraints across all of the steps while you're working on that task. And really actively maintain this as until

you're done. And if you're done, think about what is worth keeping in that from that context. So, maybe there is architectural decisions that you want to keep. So, maybe consider introducing an architectural decision record inside of your source code repository, where at the end of larger changes you always extract exactly this information and put it there. Um or derive requirements and test plans. So, you have really

a rich context here, and that's the value of your session at the moment, not just the changes. So, let's use it in future sessions. almost last but not least, there is of course then the more experience you gain, there is ways of how you can make this whole process more efficient by introducing prompt fragments or workflows. Essentially, all of the different AI coding tools offer some mechanism

to specify commands and so on. Basically, that's always just prompt templates in the end. And you can basically extract them from successful sessions and make them more and more reusable. So, that's definitely something interesting. And I would start here on a very low level. So, for instance, if there is a part in your application that you work on where you always need to basically come up with

the same API guides or whatever, this is a perfect starting point for adding those commands. Uh because that's something you can reuse without any risk. What I would really be careful about, and that's what we observe quite often actually when we work with uh companies in um in consulting them on how to introduce AI coding, uh don't basically over-engineer this and put just some third-party commands into

your project as is because uh if you watch on YouTube, there is this now new prompt library, there is this prompt library. You get just clone it and put it in your project and suddenly everything works. This will never work. Uh because essentially you make this a hidden part of your end of every session, and this has a huge impact on every trajectory. So, it's it's worth

spending time on adjusting that to your exact workflow and your exact uh context. And finally, and this is a I think a very important one as well, uh try to make this a more efficient by extending the parameter of each um I would say trajectory of the agent. So, if you think about it, essentially you always start by defining the prompt, then uh you send the agent

off to its work. It has then an inner loop essentially, and then uh you get some result back that you need to read. And the more you extend this parameter, the less you have to review. And uh that's basically what we need to optimize. So, think about uh what the AI needs to be useful to give you as high quality as possible, and uh give it to

it. Uh and that of course could be uh tools or MCPs, uh skills or CLIs that for instance allow the agent to I don't know verify if uh now the workflow that you uh implement for the user can actually be done that way by instrument by basically uh using the application like Playwright CLI and so on. But of course, keep an uh very close eye on uh

the context usage because every MCP server you add fills up by default the context uh window. And the more you fill up the context window, the less good tokens you have left for your solution, and basically the rough uh the estimate is essentially that you should be always with your entire work below 50% of the context window, which is not that much if you're already using like

30,000 tokens for just MCP definitions and all of that stuff. So, really keep an eye on that. And of course, there is solutions. So, if there if you still need for instance play right CLI, which adds, I don't know, 15k tokens to your usage, maybe extract it into an own agent that you only then use when it's really needed and so on. So, you need to decisive

there as well. And last but not least, uh I also think that's super important. Let your agents off the leash because no one really wants to read and approve all of the commands that the agent thinks it needs in order to fill fulfill a task. So, if you use cloud code, it will come up with very very specific commands that it runs in order to identify certain

interesting parts or I don't know, run test results or whatever. And when you read the fifth command terminal command that it wants to execute, you're not really adding a security layer anymore because you'll start to just click accept after 10 minutes and that really doesn't add anything. So, leave them off the leash. Of course, without sacrificing security. That means introducing proper isolation by containerizing those agents using

Git work trace so that they can work in parallel alongside each other defining basically a networking layer so that they cannot reach any any website and so on. We have internal tooling for that. There is also other solutions, but we have a tooling that is called basically Yolo Arena where you can say I want now on this project a new agent. I want to use, I don't

know, C I I or cloud code on this work tree and basically I just kick it off and then it runs in the container and when it's finished, I can always click on a button and investigate exactly this data that the agent left off. Right. Um with that, I think we are almost good in time. Um just a few words on the conclusion and I would be

very also happy if you have any feedback or discussions because obviously this is a new topic where no one knows everything about it and we are all still learning on how to apply those tools efficiently. Uh I think the most important takeaway is that AI basically is an amplifier and that means it amplifies your culture and your code and this is really the most important foundation that

you can get right in order to get the most out of AI. Uh it does not replace any real software engineering value that your team members bring into the project because this is not really something that they I can just do. They can write code very well, but they don't know all of the constraints, the histories, the judgment calls, and all of that what essentially made the

software engineers valuable in the first place. So, typically if you have a team with great engineering discipline, AI makes you faster and still maintains the quality. If you only embrace speed in your team, then AI will produce lots of code, but on the long run will make your project less quality of less quality. So, really developers need to own generated code, but also get braced for it

and that what if the AI did a bug, it doesn't matter. In the end, the one the person who the change caused the bug. But at the same time, it's bad if you say why does it take so long? Couldn't you use AI? Then it should be faster. That's also devaluing the value that the software engineer would add. So, you really need to adjust your culture to

that. And we we all should be aware and I observe this on my own. It's a transformation. It's not just yet another tool roll out, right? It's a completely different way of how we develop software because for many of us Uh, code was the process of finding a good solution. So, I felt like while I was writing code, I uh was exploring the different opportunities or different

um paths to solve the solution. Now, you take this away, uh this writing code part, but still expect everyone to find the right solution. So, we now need to retrain our brains uh to use prompting as a different way of finding a good solution. And that's a obviously a transformation. And uh type I mean, typing code was always the sugar that made us productive because we did

something, but in the end it was what's going on in our heads that really produced the value. So, with that uh it's a learning curve uh and we it's a joint learning curve. So, we need to adapt uh our value system, team alignment, tools, processes as a whole and be very uh uh curious about it. And that's I think the most important part. With that, uh thank

you very much. Uh just a few things. Uh so, uh this was just an overview of how to get productive with AI coding. There was much more in-depth as I mentioned before for an uh session uh later on at 1:30 uh given by Jonas. I think uh over there somewhere. Um and of course, it's a uh we are also very, very curious about your experiences, about your

specific project-specific challenges. So, please talk to us at your uh at our booth uh over there. Uh and uh let's start a discussion because we all need to learn about uh the how to apply AI coding in the different challenging areas. And uh we are always happy to help out with our uh experiences and learn together with you and maybe help a little bit to get started

more quickly. We provide uh trainings, workshops. And of course, if you're interested in uh having more customized AI AI agents for your specific needs like in embedded or in uh very niche topics, uh we are also happy to help you with developing custom AI agents. With that, thank you very much and looking forward to your questions. >> [applause] >> Thank you. Do you have any questions, comments?

>> Hi. >> I have the impression so that the the most important thing is the engineering discipline, right? Doing good requirements management and good change management. So, why do you in my experience, this is the most crucial part since years and and and decades. Why should we now improve on that? >> Good question. Um I think I completely agree it was the most important thing all along.

I think we should now improve on that because the speed and scale at which we can produce code now, which was maybe previously uh a little bit of a bottleneck, basically now went away to some degree. So, we produce much more code. The code bases, at least our also in the projects that we have in the open source projects, are moving much faster. So, it's now even

more important to put good engineering practices and discipline into maintaining those systems to keep them maintainable. So, I think it's They they are not not now more important, but the importance is invoked now much more often because if you go in the wrong direction and produce more and more harm you earlier. That's maybe would be my answer. >> Okay. And how would you uh convince a manager

who did not spend money on requirements management and those stuff in the in the past? >> Um so, I I would I would try I mean, I'm bad at convincing managers to just to make this [laughter] clear. But, if I would try, I would tell them basically two things. First of all, if you don't want to get behind in terms of development speed, just adding AI on

top of it won't solve your problems. What would solve your problems is putting in the engineering effort to make AI scalable because in the end AI is just amplifying what you have. If you have crap in your repository, you will have also crap in a year by just adding thousands and of millions of tokens into it. But if you invest into good practices in identifying what to

build, how you want to build it, and how it impacts your code base, then over time you get into a virtuous cycle that you always will get faster and faster and faster because AI scales all of that. That would be my my attempt to convince managers. >> I have two questions. >> Sure. >> The first one is try the try the quick. This Yolo Arena is available

for the public? >> Oh, no, it's actually not available. Not because we don't want to make it available. So, we are considering making it available as open source. But there is currently a lot of solutions that enter into the market. Many of them are focused around a specific technology like it's provided by Docker or by Anthropic for cloud code. So, they are always quite constrained. But still

it's So, that's why we built it for ourselves. But if there is enough interest, then please find us at the booth and we are happy to talk about making it open source. So, there is no, I would say uh unique economic value that we we see in that. It's just something we need, so we built it. >> Okay, thank you. And the second one is I noticed

that you when you mentioned that the longer the the chat gets, the more context it is, and the quality of the results decreases. So, you suggested to restart the conversation. How do you usually do that? Because my very naive approach would be to ask in the same chat like, "Can you please summarize this in some way that I can just start a new in another chat?" Is

that would that be a good approach? >> That's the perfect approach. It's the basically the atomic approach to it. Obviously, every AI coding agent and that's basically the sugar that they add on top. They provide certain functionality to help you with that like a {slash} compact command in cloud code or in fair we suggest a summarize button that you can click. So, maybe they help you with

that a little bit, but essentially what you're always doing is summarizing it. What I would really recommend is not just use {slash} compact or something like that, but indicate for your next step what part of this whole conversation is relevant for your next step and then extract really that because you really want to focus it down. Um you mentioned a lot about essentially we are moderating this

conversation we having with AI by uh by yeah, moderating the context and everything around it. And it seems to me that a lot of these models they become more advanced at this point already because that is being done in the background. aren't there options to let's say not keep the entire conversation of the past to sort of manage um you know, when when you have a prompt

um you get a response from the model to sort of manage the content that persists really map this in a way um that it can retrieve whatever it needs at some point, but that you don't just dump in from before and >> Yeah. So, you're right, of course. So, certain coding agents I think Copilot is for instance one of them that does that quite a lot at

least in my impression. They try to make it as easy as possible and take away this I don't I would say responsibility your responsibility to manage your session more actively by introducing some auto summarization, by being more autonomous in managing the context. And uh obviously, that's kind of helpful in many cases, especially if you don't think a lot about it as a user. But I would claim

that uh it's a trade-off. You You You uh trade essentially your intention again for something automatic, because the automatic approach of managing the session will be never as good as you knowing what you want to do next and what context is needed for that. So, I personally I dislike those auto summari- auto summarization has now kicked in, because it kind of dilutes uh the context, and I

don't even know anymore what's exactly in the context. And I think this is so crucial for the quality of the result that I would like to be in control of that. But of course, that's a matter of taste, I guess. >> If If a second question, maybe. >> Uh yeah, I think we have 1 minute 30 left, so I guess it's fine. >> What do you think

of sort of I'd call it soft moderation. Um What I've heard a lot is that sort of uh for example, specifying what role the model is supposed to take, sort of saying you are this and this. Um Or generally, another thing that I I've heard a lot is essentially, if you being nice to the model, it will present better results. >> Oh, okay. >> What do you

think of the I I'd call it soft moderation, sort of >> sort of including something like this. >> I'm not convinced that this helps a lot. Uh so, giving assigning roles is essentially always just impacting the the the style of how it replies, but it will not impact too much uh uh how it will behave. And if it does, it's very in an indirect way of telling

it how you want it to behave. So, I'm more uh I think uh convinced it's better to tell it exactly how it should behave. not give it like these soft traits uh that just influence on on which uh style it would reply. So, I don't think that this is very useful. Um that your last question was more about being nice to the model. So, I personally I

am always very nice to the model, but it's also because I just to optimize my way my way of thinking what the model needs as a context. I'm kind of humanizing it even though I know it's not a human, but I think about it if this is a person working on my project has no prior knowledge about the project. What do I need to to give it

this person? That's because I trained my brain already with to think about it this way. That's why I'm also being kind of nice to it, but I don't think that is really uh changing the output a lot. I guess. Of course, I don't know. And in the end uh who knows who I don't know how when we get used to it being really not nice to the

chat if point is spills over in our conversations to other humans. This would be a bad effect. So, I stick to being nice, I guess. Okay. Thank you very much. With that, I'll talk to you soon.