About this talk
In this talk, the speaker discusses the development and implications of AI agents designed to assist or replace software developers. He begins by addressing the hype surrounding AI, particularly large language models (LLMs), and distinguishes them from AI agents that can perform complex tasks. The speaker elaborates on AI agents' capabilities, such as tool usage, reflection, planning, and collaboration, showcasing examples of how these agents can improve productivity in coding tasks. He also introduces the Model Context Protocol (MCP), which facilitates the interaction between LLMs and various tools for enhanced functionality. By demonstrating a multipronged approach to utilizing AI in software development, the speaker ultimately argues that while AI may transform the role of developers, it won't entirely replace them; rather, developers may evolve into managers overseeing these AI systems.
Full transcript
Challenge accepted. Challenge accepted. Challenge accepted. P2 is always challenge accepted. Our next speaker is joining us from an exotic location. Madera Island, Portugal. Yes, while you were debugging in Sophia's mist mist, he was probably coding with an ocean view. >> He is the head of ML DevOps and death rail at Zen Coder with more than 15 years of experience building scalable systems, high performing teams, and cutting
edge AI solutions. He is an open-source contributor, an international speaker, and someone who knows how to bring AI out of the hype and into real world productivity. >> Today, he's going to show us something both exciting and scary. How to build an AI that will replace you. >> It it will replace me. >> No, replace them. >> Oh, that's more or less a second. So please make
some noise for >> Alex. >> All right. Hello everyone. And maybe AI will replace our hosts as well at some point. You know, virtual avatars, what not. Sorry. But yeah, uh let's talk about AI that will well it's a bit of a I guess hype uh or clickbait topic uh here but know hopefully by the end of the talk we will see uh if it will replace
you or not and we can you know also discuss this how you feel about this but before we go into this part a few fun facts about me uh in addition to what uh has been presented has been uh presented by the hosts uh so yeah been uh quite a long time in it uh mostly ML uh back end devops all that I've been building encoder AI
coding agent solution from day zero uh I've been for the last year mostly devrell which resulted in quite a lot of uh flights as you can see and um back back home in Portugal I have a few cats four cats who are patiently waiting for me uh whenever I go to to to do the talks right but back to AI and you probably seen or heard things
like that. So in this case Mark Zuckerberg predicted that AI will replace engineers by 2030 I believe and you know um when people nowadays say AI they usually mean LLM and this is not the exception. So here AI is supposed to be LLMs and as we all know LLMs are trained to predict the next probable symbol or the next probable token and sometimes it can lead to
errors. Now they are good at finding patterns but they are not that good at being grounded to physical world. So all the patterns which they see they don't really understand what's going on behind them. All right. But well we all know how AI is good with hands. So you know no not surprise here. Right. So now all those talk all those talks about AI uh replacing developers
um is not grounded just by just on LLMs but rather on uh something beyond LMS which is AI agents and we can define AI agent as a software that perform tasks on your behalf. But uh unlike any classical software which is essentially also performing tasks on your behalf, AI agents follow several design patterns or design principles. Uh first one is tool use. So agents can use any
tool you give uh them access to. So that could be web search, that could be uh running bash commands, uh file system access and so on. And they use that to gather uh information to um verify themselves to see if the code they created is running for example or not. And then they can uh go into the next uh pattern. They can reflect on this. So let's
say they created the code uh they try to run it or maybe compile it gave an error then they can reflect on this error on their work and they can um essentially fix the fix the issues uh come up in a new way and so on they can also do planning so they don't just blindly output the tokens they can u do some preliminary thinking some preliminary
preliminary planning on how to achieve the goal so if you use chat GPT or claud you've probably seen um thinking process from those models. So this is sort of like the planning um ahead of uh actually starting to output the tokens for the for the goal and then they can also of course collaborate with each other and by the end of the talk we will see how
they can um how different agents can work with together uh to create some sort of solution some sort of app for you. Sorry. And um here's a good example on how how just tool use and thinking can already extend the capabilities of LLMs. So over here GP5 is um without thinking gets only 6% on the human humanities last exam benchmark. But if you give to GPT5 access
to thinking to playing and to couple tools so Python and web search you can see that it push significantly the results on this benchmark. And if we apply if we apply the same to our to our programming world to the identity flow would look uh something like that. So let's say the task is to update the website design based on uh latest change changes in Figma. So
then the agent can first decide to use tools to get the designs from Figma. Then it might do some planning to you know come up with which files to update then generate the code. Then it might need to it might decide to sort of reflect on this work through um using tools to run the to run the browser taking a screenshot of the page and comparing comparing
it again through tool use uh with designs in Figma and then repeat this until it it is satisfied with the results may decide to also create some tests um but it's optional of course and then it basically proceeds until it uh decides that the task is done and Hopefully it is a good code which you can put in the production and yeah with that let's finally do
some coding. So I will uh go through uh several several stages several steps and by the way all the code is available on this um repository over here. So if you will want if you want to play around after the talk uh feel free to uh scan the cure code over here. Um I will give you a couple seconds. All right. So uh as I mentioned we
will go through several stages. I will skip the baseline in the interest of time. Uh but the baseline is basically LLM without any access to tools. So if you if you played around with charge PT or claudi you know that it doesn't have access by default to your local machine. It doesn't have access to internet and so on. So it can't do things like ping for example.
So uh let's see how we can on top of that how can we extend the capabilities of LLM. And for that I will be first of all I will be using open AI model right. So I will be using GPT I think I updated to five. Uh and then on top of that I will be using what is known as react framework and react stands for reason
plus act and essentially this framework instructs LLM that it needs to go through four uh separate stages. First uh the LLM whenever it receives the user query, it needs to think about what action it needs to perform and if it needs to perform any action to get more context to answer the user's question. Then it needs to actually select that action. Then uh it this section at
the post step will will will be run by us in this case right by user. uh then we will pass on the observation or the output of this action to LLM and then hopefully the LLM will have enough enough data to answer our question and you can easily instruct LLM to follow this pattern through this uh system prompt. So essentially those are the five lines which you
will need. So and it basically just describes what I've just uh talked about. So you run a loop of thought action post observation uh and I'm also passing on uh information about three available tools right so first one is ping so it will essentially as the name suggests will ping the website uh then there is bash which will run bash commands and there is web search which
will do the web search and there is example session so uh if the question of from the user would be how many islands make up madera um llm might think that it need to do a web search then the action passed passed back from LLM would be web search for example and then based on the observation it will give us the answer. So let's see this in
action. Yeah. So as you can see I'm using GT5. I have defined my tools which I just mentioned like ping bash and web search over here. They are pretty straightforward especially the web search you know nothing uh too complicated here. I'm just taking the first result of the web search. Let's check that they're working. uh and yeah all good and now let's essentially implement our um react
loop. So as as I me as mentioned before first we query the model then the model decides if it wants to do some action and then we perform the detection and pass on the observation back to LLM. if the LM will be able to answer uh the questions for us now now that it has access to some tools. So in this case it decided to use ping
uh to ping google com and it was able to give to get answer correctly. Now I can also give it access to my local machines. So for example I can ask what Python packages I have installed and of course by default LLM sit sitting in the cloud. it doesn't have access to this but uh through those tools it can get act for example in this case it
decided to run action of python-mp and it gave the gave the llm essentially access to my local environment and I have a lot of packages so it will take some time for llm to answer my question but yeah will basically summarize it and the same would go with the web search well um already this is extended cap capabilities for LM quite quite significantly. But um obviously the
way we describe tools, the way we gave tools to LLM is not very um standardized. So let's say someone else came up with a better web search. U if I wanted to switch my web search to that web search, it would take me quite a lot quite quite a lot of uh effort. So I would need to update prompts. I would need to maybe update the way
I run the actions and so on. So you know it's not very u straightforward to do. And this is where we're going to the next step. And um there are of course a lot of different frameworks to pass on tools. And actually the next talk after me will be about MCP. So I won't be spending too much time here um on how MCP works. But basically MCP
consists of two main entities. Uh MCP server and MCP stands for model context protocol. MCP server which is responsible for two main things. First one is advertising the list of capabilities or list of tools which it is able to run and then of course uh actually running those tools and then MCP host or MCP client or in this case it will be our LLM. It is responsible
for selecting which action it wants to run to gather more context from either local resources or remote resources through MCP server. And yeah so this is basically how it would uh how the flow would look like. First there is a handshake between the client and the server uh where they essentially the server gives the list of available capabilities to the MCB client and then whenever the the
user comes in with a question LLM might might recognize the need for external data um and then through MCB client and MCB server obtained that data that external data from some external system again it could be local resource it could be remote resource but then based on this obtained new context it hopefully we should be able to give you a better and this is a tool example.
So essentially uh this text will be appended to the prompt and you can imagine you having a bunch of tools um they would populate the prompt quite easily uh quite significantly by basically uh describing all the all the way what this tool what this tool is used for and uh how to run it right and yeah so there are some um marketplace but again I won't be
uh going to much details here because the next talk will be a deep dive about MCP But let's see that in action. So over here I have my uh MCP agent with MCP and now I'm switching to set 4 actually but the same the logic still say stay say the same. So MCB client here is essentially implementation of your for example um simulation of your for example
AI coding assistant or whatever you use for MCB client. So in this case I'm just um there it's a lot of code but the main parts are over here. So I'm getting the list of capabilities from the MCP server and then I'm basically again going through this react loop over here uh sending a message select detecting if the model decided to run any tools and executing those
tools and then hopefully it will give us the result. Right? So again a lot of code uh but it's essentially the same react loop as you've seen before. Then let's see this in action of one of the examples. So again here I'm giving LM access to a few a few things a few servers. First one being uh file system. Uh there is also fetch which allows LLM
to fetch web pages and there is bash to run bash commands and what this allows us to do is for example do this. So I have in this directory a docker file which can take consists two errors and let's see if the llm will be able to detect those errors and fix them for us using those the tools we just gave it So it listed the files
uh checked the checked the python code found the issue uh the first one so first of all uh the first issue was that the docker file didn't install all the packages and then the second issue uh as you can see here it tried to build the build the image and the second issue is that the base image doesn't exist so it found the two issues I just
mentioned over here and then proceeded to fix updated the code of the docker file, right? Then again try to build the build the image and yeah so the Python code inside the image is supposed to output exactly four uh four four like this. So you can see it was able to essentially fix uh what was wrong with our um image with our code in a matter of
what 30 seconds. Um so and of course you can extend this uh beyond just fixing the docker file. You can extend you can extend this to running the tests uh performing some uh browser verification and so on. So basically any tools you give LLM access to it can use to get more context and to get better and solving tasks. But of course let me get back to
the slides. Yeah and by the way if you want to implement u your own MCP server it's pretty easy to do. So here's an example of the bash tool which actually was used by LLM just now. As you can see, you only need those three um three lines of code to convert your existing code into the MCP server. But um the tools which we just used, they
are deterministic. Well, I mean they could be deterministic, but in general they are deterministic and when you run them multiple times, they will give you the same result. However, you can imagine instead of tools, agents using other agents and this adds another layer layer of complexity because now the agents are undeterministic. So if you combine one agent to another agent uh in some sort of network or
custom pattern uh any agent can go uh sideways. So you might have experienced them that yourself whenever sometimes agents can go into the loop and they can get stuck. So let's say the agent over here got got stuck or it got a wrong result. It will basically mess up the whole the whole chain of agents. Uh so putting those things in production is a bit hard still
but nevertheless let's see how they can for perform um together and we will be building a supervisor uh network over here like this. So we will have a supervisor agent or sort of like product manager and we will have a multiple uh virtual engineers working together to build an app and there are several frameworks available to build multi- aent systems. uh the most popular ones probably are
leng graph which we'll be using today uh crew AI there is also new protocol from Google agent agent so essentially you know it's much like with tools u lms there are a lot of different cho options to choose from uh but I will be using lang graph it is an open source uh library from lang chain and let's quickly jump into the back to the code this
one yeah So again I will be given access to several tools for my agent. Uh file system tool, playright browser allows it to navigate the browser, shell tool, web search and another one which I call human which essentially allows LLM to ask questions back to me to to the user and again I'm using uh sonet 4 here but the same you can be do you can do
with GP5 as well. Now as I mentioned we will be building a supervisor schema where we will have one product manager or supervisor and four engineers back end front end devops and QA. Now uh in this schema let me get back to the slides in this schema uh all the agents talk only through supervisor over here. So for that we need to have some sort of handoff
between uh the subordinate agent back to the supervisor and this is what I'm defining over here. So basically we have the transfer to all the subordinate agents and then we have also transfer to supervisor. So all the different agents will use the appropriate transfer to transfer the control back to supervisor or to supporting agent and then defining the our own um agents is pretty easy with the
line graph. So over here as I mentioned I have five um five engines right so and the prompts are pretty straightforward pretty simple in case of product manager you're a product manager you know do planning uh assign step to specific u special workers special agent and then uh pass the control back to this agent and yeah that's pretty much it and the same goes with the back
end front end and so on you know nothing fancy nothing too fancy here just you know implement a task uh create a sub plan and then pass on the control back to the product and then this is again we using the same react framework react agent as you can see it is quite uh popular and I'm passing on all the transfer to agent tools to all the
agents and then over here I'm creating my swarm of agents and they will be working for us on this is just a coloring for output so they will be working for us on building a website for a conference should have several pages um use the standard you know standard tools uh react fast API for the back end posgress docker so and so on so let's uh start
them uh I don't think we'll have time to see them through the end because it might takes them like 10 20 minutes but as you can see uh the starting end point for us as product manager which decided that the first step would be to implement back end transferred uh to the backend agent where the back end agent decided to have the sub plan for the step
one. Uh proceeded to actually implement this uh created deers started to fi to write the and it will continue to do this for some time. As I mentioned this usually takes around 10 20 minutes. So maybe by the end of Q&A session we will see the results but yeah it will still going on through the first step. Let me disable scrolling here for us to ease. Uh
yeah, still uh back end agent is u doing some of the schemas for us. So yeah, again it will take some time. Now um writing those agencies can be fun and you know defining your own agents and putting this multi- aent system in production is uh quite fun and challenging. It's quite a big uh engineering challenge. But if you want to experience uh those agents already now
in your ID or even outside of ID uh you can use for example encoder right so as I mentioned we are AI coding agent solution and uh you can through us you can define your own bunch of agents you can share them with your organization. So as you can see here those are the agents which most of them that I have been shared by with my organization
by my colleagues and you can use them in ID as well of course. So the same uh agents you can find in ID over here but uh since we are talking about sort of um future and autonomy of the agents you can use those agents in your autonomous CI/CD pipelines. So what this enables you to do is for example um over here I have a J ticket
which I can label uh like this and that's pretty much it what is needed from me as engineer to do to ask the agent to solve the ticket. This on the back end kicks off my custom one of my custom agents uh in the GitHub repository. So let's see over here. Yeah. So the agent is running already for me in the background. We won't be waiting for
it to finish because I have few pull requests already opened before. But the end the end result of this agent would be the pull request which which solves the ticket. But on top of that I have a few other agents which which can review the pull request which I can also autonomously trigger over here. So as you can see another agent came in and did the review
for me. And then of course on top of that I can address the comments with another agent. So you essentially you can create have your own multi- aentic system through custom agents which can solve tickets for you or whatever you want them to do, right? And yeah, so essentially you would be like uh these guys over here uh with CH GBT just uh carrying you all the
way. And yeah, I have I think couple couple more minutes. So let's talk about the future, right? So let's uh talk about uh if AI will replace developers or not. And you probably heard a lot the phrase that uh developers will be replaced by developers who who are using AI. So it's kind of an exaggeration of this phrase, but there is some uh truth to that because
of course AI is uh something that enables you to move faster and enables you to um work across multiple teams across multiple functions, right? So you essentially as you've seen in the multi- aent system example you could be essentially a single person handling across the whole agile team right so basically you could be a single person cross functional team but of course still you need to understand
what's going on because otherwise it can be uh quite daunting to you know uh when nothing when nothing works and you need to fix this and so yeah we still need to have some understanding of what's going on otherwise things like that happen. Yeah, but seriously um in my opinion and I think in the opinion of a lot of people AI will not just replace us. It
will be more of a u transformation technology. So it will it will definitely transform the way we work as as you see for example with the example of Jira right I me as developer as developer I did I didn't need to write the code I just had to review it and well trigger my agent and maybe nudge it in some way if it gets stuck or help
it in some way if it gets stuck but I didn't need to do coding I didn't need to um well do anything pretty much right I just had to review the code and maybe manage my um agents. And this is what the future potentially will look like in probably a couple years, right? Where we as developers, we will become more of managers of virtual agents rather than
um writing the code ourselves. And with that, yeah, so there is also of course uh one other job potential which uh can be still left if you want to write code, right? Because yeah, I need a lot of data to write the code. But yeah um that there's a few QR codes for you. So on the this QR code over here is if you want to try
some coder if you want to try agents uh over here you can scan this one and this one is for my contact details. uh if you want to get in touch and I think we are almost at time so I will um I will leave your codes for a few seconds and then I will put back my multi- aent system while I'm answering your questions if you
have any and as you can see in the meantime already front end agent I think kicked in yeah you can see there's front end agent already working right so yeah happy to answer any of your questions Any questions? >> Do do we have questions from the room here from the >> or as you know, Alex will be waiting for you in the speaker's corner during the next
break. So there are no questions. H there is one question please. >> The guy who wants the shirt. >> Hello. >> My name is Matias. from a QA. And uh with that in mind, I I can imagine for this to work with just moving the Jurro task, you need a lot of context for for the AI to understand what needs to be done. And I can imagine
the Jira task also needs to have a lot of context >> for it. It understands exactly what needs to be done. How how to deal with that? It's like it will consume from a background or >> Yes. So context is indeed an important part and well, you know, the Some of the recent models they have a large context window right and at the same time it seems
that agents nowadays can successfully manage their own context and they can successfully select which files or which context is important for them and they can discover that on their own. So uh you know one year ago a lot of people were were investing into rag for example right and nowadays at least in the coding world we see that you don't need rag for agent to be able
to solve the task efficiently it just can find the ne necessary files on its own using tools like grab you know search whatever and this allows also you as a user not to worry about the context because the agent will sort of self kind of self-manage itself um and yeah but yeah of course depending on task depending on your uh code repository. Sometimes you need bigger window
bigger context window and there are of course models which can handle this and this is something u that goes kind of in the in the multi- aent system because sometimes you want uh some agents work on one model right for large context window so let's say uh one one agent can use Gemini to prepare documentation for example or prepare spec because it can fit more data into
the context window but then you might want to have sonet because it is better at coding uh to go and and start with this spec because already with the work with the work of finding the right context was done for this agent right so you can also sort of mix and match uh those things and I think in the meantime the website should be actually working if
I can get my screen back to this uh yeah so as you can see in the meantime the agents created the website for us they are now actually testing it. So over here the Q somewhere the Q agent was I think testing it. I just saw it somewhere. Yeah. So Q a agent is now testing it. So it probably submitted a few things. Let's see. Vox. Yeah.
As you can see Q agent in the meantime submitted the talk uh for the conference website. I think there's a question also >> but please make it very quick. The next >> Yeah sure. Uh my question is about uh handling uh legacy code like uh what is your experience with AI AI agents like uh I see they are uh quite uh nice writing new code but uh
what about legacy code? Do they handle it well? Do they need a lot of context? >> Mhm. Mhm. >> Or do you think that the better way would be just to give them uh requirements and uh they will do anything a new uh yeah legacy code um is quite can be challenged can be challenged indeed uh depending on the code itself right um but one of the
one of our clients actually used us to convert from Java 1.6 six uh quite old Java right to Java 21 I think. So you know again depending on the quality dep of the code uh agents usually can work out what's going on because if you give because they can uh for example try to run the code they can try to uh do something around the code and
they that can help them understand what's going on. And of course you don't if you if you're talking about you know refactoring legacy repositories you don't want to just try to oneshot refactor the whole thing. you want to do this s sort of step by step or file by file and then in that case um LMS are usually um already good enough to understand what's going on
or if they can't understand the transform text they can potentially again use the backer use u just run the code execute the code and this also can help them understand what's going on and then transform the the legacy code into the newer version or newer language whatever you want to what have to do >> thank Thank you very much. >> Thank you. >> Alex will be waiting
for you in the speaker corner during the next break. And now let's send him off of the stage with big round of applause. ALEX
