From Code Completion to Autonomous Feature Delivery, Ioannis Kolaxis

In this talk, Yiannis Koulakis discusses the evolution of large language models (LLMs) in software development, emphasizing their increased capabilities compared to just a few years ago. He uses the metaphor of a baby elephant, which, once believing it couldn't break free from a rope, symbolizes the outdated views some developers hold about LLMs' abilities. He details how models like Claude Opus have changed the landscape by performing complex tasks that previously took human developers hours to complete. Koulakis highlights the importance of effective onboarding for LLMs, especially with legacy code bases, and outlines a structured approach to leveraging LLMs, including exploring code, planning tasks, and coding efficiently. He stresses that coding is evolving to focus more on problem-solving and context management rather than direct code writing.

Let me start with a story. When a baby elephant is born in captivity, the trainers would tie the elephant to a rope, tying the elephant to the ground so that it cannot leave. The baby elephant will start pulling the rope using all its strength, pushing against the rope so that it breaks free. And it tries for days, weeks, months, but it's still not strong and therefore cannot

break the rope. Years pass and this baby elephant has grown and became one of the strongest animals on on planet Earth. It can pull trees out of the ground. However, it has stopped trying to break this rope. Why has this happened? Because the elephant stopped believing that it can break the rope. It's not trying anymore. It's convinced that it's it's not strong enough to break free from

the rope. Has any of you used any LLMs a couple of years ago to try how this work in software development? If you did, then you likely experienced that LLMs could only generate three or four lines of code, some small code fragments. And while this was impressive back then, it couldn't do any significant software development work like developing new features or fixing bugs across multiple files. This

was like the baby elephant. The LLMs were not strong enough. However, 2 years later the LLMs are more capable than ever before. They are the big stroke elephant that can solve complex software problems fix bugs across multiple files or build new features. However, many of us have formed that belief that LLMs cannot do any significant software development work. The models have changed. And you can see here

on the graph how they have evolved in the past months. GPT 3.5 2 years ago could barely solve tasks that a human developer could solve in around 30 seconds. GPT-4 could solve tasks that a developer would solve in 4 minutes. That was back in March 2023. But look what happened suddenly. In February 2026 Claude Opus was released, Opus 4.6, that can solve software engineering tasks that would

take 12 hours for a human to solve. This exponential growth in the capabilities of LLMs took place roughly in the last year. So, if you go back to February 2025, the most capable model could only solve take 1 hour for a human to solve. What is interesting here is that the models have significantly changed. The question is, have we changed how we do software development work? Have

we realized how capable the models are today? My name is Yiannis Koulakis. I come from Athens, Greece. I work for Accenture for the innovation organization of Accenture, where I specialize in using AI to build greener code, code that is more efficient, uses less energy, and has lower costs. I was speaking here in JPrime 2 years ago back in 2024, and my motivation today is to show you

how many things have changed since then in the in the software engineering space. How big changes are happening thanks to the LLMs. Is anyone here working for a legacy code applic- for a legacy application? Please raise your hand. Raise your hand if you're working. Well, I have some bad news for you. LLMs do not perform well with legacy code bases. And the reason for that is that

first, they would have to understand the code base well before they can provide solutions, before they can provide fixes for the legacy code bases. According to a study from Stanford University, the productivity increase from using LLMs with legacy code bases is minimal. When we try to solve complex software engineering tasks, the productivity becomes even lower. And when you combine that with legacy programming languages like COBOL or

niche programming languages like Haskell and Elixir, then the productivity becomes zero or it can even become negative. In these cases, the LLMs can actually slow down developers instead of speeding them up. On the contrary, if you're lucky enough to work for a greenfield project writing from scratch, this is where LLMs excel. They can boost productivity by up to 40%. This becomes even better when you solve simple

tasks that are well-defined and when you're using popular programming languages like Java and Python because most models have been trained with plenty of data on these languages like Python and Java. Three key things to remember here. These are the three things that differentiate the performance of LLMs. First question, how old is your code base? Second, how complex are the tasks you're working And third question to ask

is how popular is the programming language that you're using. Imagine that you hired a new developer. Top talent, the most experienced developer ever. It's their first day at work. What would you do? Would you ask them to start coding right away? To start implementing the next feature? Of course not. You have to train them first. You have to explain them how your application works. What are the

business requirements behind the application? How What is the architecture, the coding conventions? There are so many things that you need to a newly on-boarded developer. The same thing happens with LLMs. That's why LLMs struggle with legacy codebases. We have to on-board LLMs. We have to tell them what our legacy application is about, what the architecture is. Before we move on to how we on-board LLMs, how we

tell them about our application, let's think what we do with new hires. We need to explain them first of all the business behind our application. How are the end users using the application? What are they using it for? Then we need to explain how the code is structured. What are the different software components? How do these components interact with each other? What are the coding conventions? And

then we also need to explain about the pipeline. What is the build pipeline, the deploy pipeline? How do we monitor our application? That's what we need to do also with LLMs. We need to provide them the context about our application. And we have two different types of context. We have constant context and dynamic By constant context, we mean the things that usually remain constant, such as the

architecture of our application and the coding conventions. And by dynamic context, we mean the things that change depending the task at hand, the task we want to solve. If we want to refactor some existing uh software feature, we need to provide that in the prompt, explain what we want to refactor. We need to point to the most relevant files that we need to change. And we also

need to provide the conversations, all the conversations we had with the LLM so far. Here is the funny thing about LLMs. The LLMs do not have any memory about the conversations they had with us. So, every time we provide a new prompt to the LLMs, what happens in the background is that the coding agent would also share all the history of all the previous prompts we provided

and the answers, the outputs of the LLMs to our prompts. That's how it works. Therefore, we have this constant context that is sent with every single prompt, and we also have the dynamic context with which changes depending on the task at hand. What's an easy way for us to generate that constant constant context? When working, for example, with Claude Code, we can run the init command, which

would scan our code base, understand what the architecture is, what coding conventions we use, how we run our application. It would figure out all these things on its own, and it would generate a file, a markdown file known as claude.md, which contains all that information. That's the most convenient, the easiest way to create this static context, which would be included in every single prompt after that point

in time. That's how this happens in action. This is the code repository of my personal website, nothing too complex, and I'm running the init command in Cloud Code. It scans the code repository. And after 3 minutes and after using almost 5,000 tokens, we have the cloud.md file generated. You can see on the top it figured out that this is a Jekyll 4.3 static website. It provides some

commands on how to run that website locally, what the architecture of the website is and some other additional information how the responsive images are being automatically generated search engine optimization is. So, quite a few details about the application. By the way, this is my personal website. And as a demo, we will try to migrate my website from jQuery to using vanilla JavaScript. So, here is what happens.

Whenever someone navigates to the website, the browser would automatically download jQuery as a dependency. You can think of it as a big library that contains many functions. However, only a small fraction of that library is used. To improve the performance of the website, I wanted to replace that big library with native JavaScript code only for the functionality used by the This is something that I tried quite

several several times in the past 3 using the traditional approach of using my bare hands, then using some of the early LLMs, but I always failed every single time. Would this migration be successful with the latest models? What do you think? We'll soon find out. What is very tempting today with all these coding agents is that we can easily provide a prompt and ask from the coding

agent to start writing code right away. So, the most tempting thing here is to go to Cloud Code and tell, "Please migrate away from jQuery and use vanilla JavaScript." However, from my experience, this is the least effective way of working with coding agents. The most effective way is to apply this three-step workflow: explore, plan, and code. For the given example, the first step is exploring the code

base. In which parts of the code are we using jQuery? And if someone wanted to test the jQuery functionality, which tests would they run? That's quite basic. And after running that step, we just get a file with the summary of the findings. Then in the next step, in the planning we ask tell us how it would replace all the jQuery calls with vanilla JavaScript Not every change

is the same. Some changes are more straightforward than others. Therefore, we also prioritize these changes from lower to higher risk. And then in the third step, we ask from the coding agent to actually apply that plan making the code changes to the files. Here is the first step. We're double-clicking into the first step. That's the prompt we will provide. Find every jQuery call in the code base,

create the list of tests that a user would run to test that jQuery works, and then save these results to a file called explore.md. That's the prompt. Running in practice, it takes some time, after waiting for 3 and 1/2 minutes, and after spending 28,000 tokens, we get the explore markdown file lines in the code where we use jQuery, what each call makes in plain language when someone

visits the website, and then we also get some plan on how to replace these uh with native JavaScript code, and some prioritization regarding the risk of these changes. So, second thing is the planning. So, we map each jQuery call to vanilla JavaScript, we rank these changes by risk as I already mentioned, and we include also the manual tests. So, after minutes and 41 seconds, and after consuming

26,000 tokens, we get the detailed plan for all All right, we've done the exploration, we've done the planning. Now, it's the part that we are almost familiar with, the coding phase where we execute these changes. We ask from the coding agent to apply these changes and after waiting for something more than 1 minute and spending some tokens, we can see eventually the changes in the code base.

We have removed jQuery as a library and we have changed the file, the custom JavaScript file that previously used jQuery and replace that with vanilla JavaScript. For each change, you can see the code and also you can see how risky that change was. So, it's not just a couple of lines that changed in the code, but it's a significant number of changes. And now comes the critical

question. What do you think? Was the migration or not? We would have to test. I'm testing the website as a mobile and when I scroll there is an error thrown in the console. Migration was not successful. If we zoom in, we can see that the error in the console there is an index size error, failed to execute arc, the radius provided is What do we do next?

You should all know by now it's the explore, plan, and code phase. First, we ask from the coding agent to explore the error, explore the code base, then create a plan on how to fix that and only then after we've done the explore and planning, will we proceed to the coding phase of applying the actual changes to the code. And after iterating a couple of times on

fixing these errors, we can see that the performance in the website increased from 90% to 92%. It is some kind of improvement. Remember this slide with the three steps? Where did we define the tests? We define the tests first before making any changes. This is very critical when working with coding agents. We should always start with tests, and that's not really anything new. We were talking about

test-driven development for ages. It turns out that test-driven development is more critical in the age of AI agents, of coding agents. Here is what happens. Why should we always start with the From our experience so far, if we do the opposite, if we write the code first and then ask from the agent to write the tests, it's quite likely that the agent will hallucinate. It generates the

tests in such a manner so the tests would pass even if the code is buggy and it has defects, it would adapt the tests so that the the tests the tests would would pass. So, the agents tend to cheat when generating the tests after the code. However, if we follow the best practice of generating the tests first and carefully review these, then we can use these as

some sort of safety net when generating in a second step. And that's also a practice that is emerging by many developers while working with agents nowadays. So, what many developers will do is that they would spend most of their time on defining the tests, ensuring that the tests are and they would only review the tests. As long as the generated code passes the tests, why should we

care about the code? So, there is a shift from reviewing code to reviewing tests. That's what we see nowadays. Remember, as we worked through the explore, plan, and coding phase, all these numbers I told you about, the tokens consumed, well, there was a reason behind that. It's very critical as we work with coding agents to monitor the use of the LLM's memory. The LLMs do have a

working memory that we usually refer to as context, context window. And here you can see how the usage of that memory increased across three these three steps, from 28k uh 28,000 tokens at the end of the explore phase to 62,000 tokens at the end of the coding phase. That's how we can all check how much of that context window we've used at any given point in time.

We run the context command in Cloud Code, and you can see that in this example, after running the code step, the code phase, uh we've we've been using 31% of the coding of the context window. And that's the progression across the three steps in our example. How we moved after the first step from using 14% of the context window to using 27% and and then using 31%

at the last step. As a rule of thumb, we should uh be striving to stay behind to stay below the 40% While you do see that there is one the potential to use 100% of the context window, when we use more than 40% or 50% LLMs tend to be more inefficient. It seems like they're drowning in too much information and they are unable to retrieve previous instructions

that we provided to them. Has it ever happened to you working for a long period with an LLM exchanging prompts back and forth and asking for the same things again and again and the LLM ignores you? That's that's usually the case when the context window fills up with information, LLMs tend to get lost. That's the dumb zone. You should always try to avoid So, the the rule

of thumb is to stay from 0 to 40% of the context window. When going above that, you are in the dumb zone. What happens if you accidentally enter the dumb zone? What are your options? How can you fix that? There are three ways we can fix that. First of all, the most obvious one is to clear the context. We remove everything from the memory. We you know,

we we delete everything. the the we could use that when we are starting a new task. We finished the previous task and we want to start clean without having any information in our The second thing we can do is to compact the context. By compacting, we mean summarizing the context. We usually provide the whole context window to another LLM and create a short summary of the discussions

we had so far with the LLM. That's not the best solution because we do lose some information when creating that summary. However, that's the only thing we can do when having long-running tasks, tasks that are more complex and take more time for the LLMs to solve. And that's also one of the most common problems. As the LLMs become stronger, we give them more complex tasks to solve

and therefore fall into the trap of using more of the context window, being forced to compact that context The third option, which is relevant relatively new, is delegating independent tasks to sub-agents. Imagine you have the main agent, which is sort of the orchestrator, and when you have to perform an independent task, for example, exploring a code base, you delegate that task to a sub-agent. That sub-agent has

its own context window, works independently, and after they've researched the code base, they can return a a of their finding their findings to the main agent to the orchestrator agent. Remember 2 years ago, everyone was talking about prompt engineering. Now, context engineering is what everyone talks about and this is why I've also been talking you about the context window. As we get bigger tasks for more capable

models, it's more critical than ever to carefully manage the context window. It's one of the most critical assets that we have. Here is the definition from Andrej Karpathy on what context engineering is. It's a delicate art and science of filling the context window with just the right information for the next step. It may sound simple, but in practice, it's not that simple. What is the right information

that you need to provide to the LLM? And what is the next step? And when you have a big task, how do you break it into smaller tasks that the agents can handle? How do you put them into sequence so that you have the previous step and the That's essentially how our role as software developers is transforming. It's becoming less about coding and it's becoming more about

thinking. First of all, thinking about the most important problems that we need to solve. Then, it's about how we break these big problems into smaller ones so that they fit into today's agents. And then, it's not really about coding. It's about following a structured approach of exploring the problem, creating a plan for attacking the problem, and only then generating the code for solving that problem. And when

doing that, we need to write the tests first. One more dimension that is changing our profession is that it's becoming less about coding and it's becoming more about experimentation, experimenting with new things, with new ideas. Personally, I've been having more ideas than I was ever able to implement. And now with uh all these agents, I can implement more of my ideas. I can experiment freely bringing these

ideas into life faster. So, that's a nice experiment that we could try together. One of the challenges I have in conferences is that there are so many interesting sessions, and I don't really know which of the sessions to attend. One of the practices that I follow is searching on YouTube for previous recordings from the same speaker, from the same session, so that I get some sneak peek,

some preview about what they're talking about, and whether this is something that would be of my interest. So, how easy would it be to take J Prime's conference agenda and automatically create a new agenda that includes the videos from all the speakers for the sessions they are presenting today and tomorrow in J Prime? Should we try that? So, here is the prompt. As part of the planning

phase, here is the prompt we can provide to a coding Build a Java application that enriches the conference agenda with videos from the same speakers. So, first step extract the agenda, all the speaker names, and the session titles. Second step, search on YouTube for the same session from the same speaker. And then the third step, generate the new agenda. I did provide that prompt to Claude Code.

You can see that it it took me some time, more than 12 minutes, and also consumed 44,000 tokens. That's why I'm not doing this live. And after 12 minutes, this is the code, the Java code that it generated. So, first step it extracted the existing agenda from JPrime. Then it searched for relevant sessions on YouTube. And then the third step was rendering the HTML. I can show

you the code if you're interested after the session. And here is the new HTML page, the new conference that was generated after 12 minutes, literally 12 minutes. It's the same agenda, but it includes the links to the YouTube videos from all the speakers. So, it's not And today's LLMs are not just about generating small code snippets, code fragments. They can do more impressive things. And whatever idea

we have, we can easily implement it. You can also access the enhanced the enriched so agenda here using this QR code. You can see it live. So, that's just a sample of what you can do with 12 min 12 minutes of running time with your coding agent. Closing the session, if there is one key message that I want you to remember is that the rope cannot hold

us anymore. There are three things that you can try. First of all, try the latest models, the most capable models. These are usually models that are not available for free. You would have to pay to try these However, it's definitely worth the effort because these tools are completely rewriting the rules of software development today. And these changes only took part in the last 6 months. For most

of our industry, we still don't know how these changes translate >> [snorts] >> what these changes mean for our profession. The second thing you can do is to mind the context. Just like you would onboard a new developer to your team, you need to continuously think about the information you that you provide to your LLMs, how you onboard them to your code base, and how you actively

manage their context windows so that you don't go into the dumb zone. You should always stay in the smart And the third thing to remember is that coding, you know, is is becoming less of our profession. It's more about thinking the next problems that we will solve, how we will break these problems into smaller tasks, and following a structured approach of exploring, planning, writing the tests first,

and then moving on to closing this session, there is one question I have for you. What will you build next? What amazing things would you build? You can use the microphone to share ideas or ask questions you have about Is there a microphone? Really, for for the audience, do you have any questions or any ideas, something that inspired you about implementing next? >> World Cup starts soon.

There'll be matches in different time zones, so it one idea I have is generate an app >> [clears throat] >> that will scrape this data and provide like real-time or what happened yesterday in the morning, so I can not like have to visit different sites, but uh this to be generated for me. >> You mean for code generated by other teams in different time zones? What What

>> The results of the matches, highlights, who scored goals, etc. >> see it. >> Yes. >> Nice. Nice. Nice idea. Does anyone else have something to share or to ask? >> I want to build uh an AI uh project that is going to uh follow my CI CD in Jenkins and uh in case of failures to like um notify me uh and even fix uh the build

if it fails. That's That's possible. For example, I I I was looking recently Cloud Code released a new feature called Loop where you can provide a prompt to do exactly that. It's like scheduling a cron job that would continuously check your CI/CD pipeline and if it fails, if there are any issues, it could even troubleshoot what is wrong and notify the the the right persons who could

fix that. So, you could definitely try something like that without spending too much effort. >> Also Also one other I idea is uh because Elasticsearch uh is uh full of so many um messages that uh it's uh uh very hard for human being to process Uh maybe we can use AI again to uh filter them out and uh alert as well with the most uh uh important

uh to be uh seen at the right moment. >> Yeah, uh everything is possible. You could delegate this entirely to an LLM to analyze continuously analyze the data. The drawback of this approach is that it would it may be costly. It may be consuming too many tokens. Or the alternative approach would be to write code that automates this process. So, you only consume tokens once to build

a tool, to build some sort of an And then you could use that all the time running, for example, in production continuously analyzing your elastic search logs and act on these logs. So, the traditional way of building tools you but this time you don't really have to write the code yourself. Is there someone else? >> Uh I already built an application and SDK that are signing the

picture of the time of capturing it on the phone. So, uh that guarantees that it's not AI modified or generated. >> Okay, whenever you take pictures, you mean? >> Yes. At the time of the It doesn't save it on the device. Uh it signs it before it saves it on the device. And you register the create when you register your device creates a public and private key

register your public key on the server. And after that, when you take the picture and it's and it uh signs the picture, anyone can verify that it was taken on your device and it was on on that time which the time also comes from the server signed, so you cannot fake the time, you cannot uh uh you cannot fake the uh GPS coordinates if you allow them.

>> Yeah, it's a cool cool feature. So, you're actually signing the photo as if it were a document. >> Yes. So, that guarantees it's not a photo that you generated on AI and ship it to someone. >> Yeah. Do you also the Oh, okay. Okay. Uh I have a quick question. It's Oh, okay. Yeah. It's not related to some plan for a project, but uh you said

this 40% after that it's a dumb zone. Uh how do you typically clear the context? Do you clear it or do you compact it? >> Yeah, yeah. >> Because if you compact it every time, then at some point uh you'll just have >> Yeah. Uh I have So, speaking about this uh framework of this workflow with the steps, uh imagine that from the very first step, the

explore step, we were uh on the 40% uh boundary, you know, towards the the dumb zone. Okay? So, what could we do there? Uh one best practice is during the explore phase, the first step, we create uh with the findings from the exploration phase. In the next step, we begin by clearing the context, so we have zero memory used uh in the context window, and we provide

as input the file from the previous step, which is like a summary with uh the bare minimum information you would need. You provide these inputs to the LLM, and then you generate another file uh with your plan, again as a separate file. And then in the third step, in the coding step, again you clear the context and provide only what the model needs to know. You provide

the plan. And based on that plan, it will generate your code. So, this is a detail that you can use together with this three-step approach, with this three-step framework. Clearing the context without losing the critical information that you need for each step. Does this answer your question? Thank >> Yes, there there is. Um with the context of MD files, what do you think about files which hold

context about teams? >> Sorry, I cannot hear clearly. >> [gasps and laughter] >> Uh what do you think about MD files that hold context about teams? So, you have context of team that's working on a a whole thing, like a whole product. And there is no other documentation, and you kind of gather the information from people, basically like human word. And you try to kind of make

a really big file with all the information that you have gathered, essentially. So that you can have a tool that's specifically helping this team in their context. >> Yeah, is it helping the LLM though? So, going back Okay, to this slide. If you create uh a file that is too big, this will be sent with every prompt. And whenever you start, it may be the case that

you you are already consuming most of your context window, leaving no context window for doing useful work. So, it it is not about providing too much information, but providing only the right information that you need for the given task. Even when talking about the constant context, that static context which is general information for your application. So, you shouldn't be providing too much information. The best approach is

the explore phase that I mentioned. That's why it's the first step. You ask from the coding agent to explore your code base for the given area where you want to implement a And then you proceed from that. Or an alternative practice that some teams are applying commit in their code repositories many different files. So, you have multiple software components, multiple uh folders in your code base. And

for folder in your code base, you could have a different file providing the information necessary for that software being located under that folder. So, this is also an alternative practice. >> What about skills then? Can we skills? >> Skills. Um like in some kind of a code tool, you can define skills or persona, so to say. Is it then better to define this kind of context in

there? >> Yeah, you you could for sure define new skills. But the more skills you add, the more space you use in the context window. And the same happens also with MCP servers, which were trending a lot, especially last year. Uh what many professionals, many of our industry, have found out is that the more you are embracing trends like MCP, skills, and the more context you're adding,

uh the faster you get to the dumb zone, rendering the the LLMs ineffective. >> Thank you. >> Yeah, that there is no single established practice. Uh, everything is moving at a rapid pace nowadays and you know, it's up to us to experiment and see what works best and then come up with industry best practices. There is a question there. >> Hello. Do you have any tips on

between the explore, the plan, and the code on changing the model or changing the version of the model used? again, there is no golden rule here, uh, but you know, for example, for Claude, you have the most capable model, which is Claude Opus, and it's also the most expensive model. Uh, the planning phase, you know, as a practice, most developers prefer using the most expensive and most

capable models for the planning phase and then using the smallest and less capable models for the coding phase, for example, Claude Sonnet, which is cheaper and faster when you write code. However, there there is not a single approach. The best thing you can do is try what works for yourself. What tries what works for your code base because there isn't a single approach. It Are you working

on a legacy code base or greenfield project? Uh, what kind of language are you using? Is it a popular or a legacy niche language? And what task are you trying to solve? So again, we go back to the study from the Stanford University. There isn't a single approach. It will all depend on your project. >> Hi. Thank you for the presentation. It was really interesting. Just to

ask for those 40%. How did you find the number? Is it really strict? And is it applied for all >> So the the number 40% is not strict. In other presentations or blog articles, you will see someone mentioning 50% or 60%. It's a rough number. There is no science behind it. It's more what we found out from experimentation. >> However, if you search for context you know,

generally about how to manage context, you will find plenty of information from multiple sources. There a single authoritative source that says that 40% is the golden rule. >> Actually, I myself want to ask one question. Yeah. What would you say is the best way to clear context between the planning phase, the coding phase, the whole idea cleaning context. >> New task. New New chat window. >> There

isn't a a single best way. It will all depend on the task that you want to solve. So that's here the the second column, when to use each method for optimizing the context. So, when you when you decide to start a new task, you should always clear your context. That's the first case. Uh if you have a long-running task, then there are two ways of handling this.

Either revising the task that you've assigned to the model because it's too big, it's taking too much time to solve it, and it's already filled up the context. So, you go back to the and create a different plan of how to break the problem into smaller problems, into smaller tasks. So, that's number one. And number two is what is mentioned here on the slide. You compact the

existing context, essentially creating a lossy summary of that context. There is the drawback though that you may lose valuable information for the task at hand. So, it's not the best And sub-agents are the newest technique, trending a lot. You could also use the sub-agents technique, but this is again for independent tasks that you to a sub-agent to work independently on that task and then return the answer.

So, again, the challenge is how you break tasks bigger tasks into smaller independent tasks that you can then later delegate to sub-agents. >> You're welcome. I'll be around in the conference if you have any questions and are feeling more shy, you can uh always come to me directly and ask the question, okay? >> [applause]