About this talk
In this session, Abdel, a developer advocate at Google, discusses his experiences working on legacy applications using artificial intelligence tools. He presents a case study involving a Swedish company that struggled with a monolithic Ruby application comprising 100,000 lines of undocumented code. The talk emphasizes the challenges of maintaining legacy systems and the cultural inclination towards shipping new features rather than refactoring existing code. Abdel explores the use of AI tools to better understand the existing codebase instead of rewriting it line by line, highlighting the importance of Test-Driven Development in ensuring the new system mimics the old one. He shares insights about optimizing processes through the transition from monolithic to microservice architecture and underscores the evolving role of developers in an AI-driven environment.
Full transcript
Hi everyone. Buenos días. That's as much Spanish as I'm going to speak today. I also know a very long phrase in Spanish that's a friend of mine taught me a while ago, but it's a very bad one. So I can tell you later in private. I can't say it on stage, especially if this is recorded. Um So my name is Abdel. I'm a developer advocate at Google.
I've been in the industry for a while. I've been doing a lot of things. I'm seeing save ambassador and um I live in Sweden and I'm from Morocco just south of here. Um just few kilometers south. As a developer advocate we we we get to do this. Like we get to do talks and stuff, but one of the things I actually like is working on real cases,
real, you know, projects. And uh what I'm going to be telling you over the next 30 minutes or so is um what I learned from real projects I actually worked on uh with AI. and because I like analogy, that's my kind of teaching side, uh I came up with this thing open sesame. I I think you know probably the Alibaba and the 40 thieves, right? Everybody knows
the story. Okay, cool. All of you know probably. every every company, every enterprise um has some sort of cave of wonders I call it. Your cave of wonders is a legacy application that somebody built a long time ago and you probably don't know how it works. You just, you know, deal with it. It's just there, it works, it's happening and you know, we have this um this
uh kind of uh culture in our industry which is if it if it's if it's not broken, don't fix it. If it ain't broken, don't fix it. Which basically means if things works, just don't touch Um because most of the time actually most people or most developers what they want to do is they want to ship new stuff. Almost no one wants to refactor existing Um I
don't know if this is actually a thing in the industry, but at Google at least um no one actually gets promoted for fixing [ __ ] Everybody gets promoted for shipping stuff. So, if you spend a month refactoring library, no one is going to give you a promotion. But, if you ship a new button, then you get promotion, right? Um, so we got approached by this company. Uh, it's
a Swedish-based company. I'm not going to mention the name, but they basically said, "Hey, we have this existing application. It's a 100,000 lines of code on Ruby." And none of the company none of the developers in that company write Ruby. Came from an acquisition. They acquired an existing company. They had this, um, this code base. Um, it works. And they just like did dealt with it, right?
Uh, so 100,000 lines of undocumented Ruby code. The worst. So, it's both code they don't understand and it's also undocumented. And, um, it's a monolithic application. And what this application does basically is that it's it's So, the company is a content distribution company. So, they dis- they distribute content, PDFs, files, audio, video, stuff like that, right? So, they get, uh, the content from the producers, the publishers.
Uh, think about the the the the companies that actually make the content. And then they distribute to the to through their platform to their customers. So, one of the things they have to do is they have to ingest all these contents. And then they have to process it to make it available to the users. So, that could be things like audio files, which could be extremely long
multi- Think about audiobook. Um, an example that we actually had in this particular, uh, project is, uh, The Time of War, which is one of the most popular audiobooks. It's about 80 hours, uh, of uh, of audio content. to process this particular file, it took their existing system 80 minutes and it costed $80. Um, the 80 80 80 is not a coincidence. It's just like a particular
example. But, it could be any sort of of of thing. But, the but the the much larger problem they had is that they they actually, when they they make their application available in new markets, let's say they enter the Spanish market, they actually have to publish even more content. They have to create a catalog, right? Which means that their system has to suddenly process hundreds and hundreds
of thousands of audio files and to make it available in this particular new market. So, they had the system and they basically acted as a wood chippers. So, as developers dealing with existing code bases, legacy applications, you are a wood chipper. You are just basically hacking at an existing code one Jira ticket at a time trying to make sure it works, trying to make sure it continues
working. The leadership, the engineering leadership in this company, they decided, well, this this is not going to work. And the main problem also is that this monolith application, when it fails, they have to restart it manually, right? Like there is no automation. Uh so, the engineering leadership decided, we need to fix this. So, they they sit down and they said, "Okay, how much would it take to
rewrite this entire application?" And they estimated it's a two full developer per year for 9 months to rewrite the entire system. So, they have to take from the existing pool of developers two people and just task them with rewriting the entire system, right? For a year almost or 9 months. Um and we all know that doesn't work, right? How many of you have been on these projects
where there is a system that exists? Here, one person, two. And then three, four, five. Okay, more hands. And then some of your colleagues are trying to rewrite it at the same time, but you have to keep it working at the same time, right? And the problem is, as time evolves, needs evolve, business needs evolve specifically. So, it's not only a rewrite, it's more almost all the
time a rewrite and evolve, right? You're not your system is not frozen in time because as you are rewriting the system, suddenly your marketing team will show up and they're like, "Oh, you know what? It'd be nice to have this new feature, right?" And then your commercial team will show up and they're like, "It'd be nice to have this new feature. So, it's almost never a complete
re-architecture. It's a re-architecture and feature addition, And most of the time actually as developers, you can just push back and say, "No, no, no. Like talk to us in 9 months. Let us do our thing and then we'll just not It's not going to work." So, basically, we came up with this um model where we said, "Okay, we have AI tools." And AI tools are actually very
good at producing new code. And you probably have heard of this term uh code is becoming commodity. You know, AI can generate new code, so we don't really need developers, blah, blah. So, we said, "Let's Let's Let's see how these tools can actually work on an existing system." And specifically, what we wanted to do is we didn't want to task two developers to read the existing system
one line at a time and then rewrite it one line at a time because that would take 9 months, Um if you are familiar with the uh Alibaba and the 40 Thieves, there are these two main characters in Alibaba in the in the tale. There is Cassim, who is Alibaba's brother, and there is Marjana or Morgiana, Morgiana in Arabic. So, Cassim is a wealthy, rich person who
is the brother of Alibaba. And when Alibaba found the cave, Cassim entered the cave, got lost in all the gold he found there, and then eventually forgot the keyword to exit the cave. And then was stuck inside the cave, and then the the the the the thieves found the found Cassim. So, that's the equivalent in our analogy of rewriting applications one line at a Morgiana was more
clever cuz what Morgiana did is that she actually tagged the exits of the the cave with multiple to to like a confuse the thieves for where they are. So, she was tagging the different uh exits. She was exploring different paths, right? So, what we did is we said, "Okay, instead of rewriting this application one line at a time, let's follow the path of using AI to explore
how the existing application works." So, we took anti-gravity and Gemini CLI, it doesn't matter what the tools are, and we integrated this into the existing system. So, we took the existing code base, we took the CLI, and we ran the code base against the CLI. This can be done with Code Act, with Cloud Code, it doesn't really matter the tool. But, the whole idea was instead of
using the AI to actually write the code, we used the AI to understand what the code does, >> [snorts] >> Most Most importantly, we used the AI to explore what we call the CUJs, the critical user journeys. So, explore all the code paths and document what the code actually does. So, the output of the AI was not code, it was just documentation, And then, using So, and
then the next step was starting with TDD, test-driven development. So, instead of actually directly from the documentation starting to write code, we wrote tests first. We generated them with AI, and then we verified them by a person, but the whole idea is write code that can verify that the new system, which is not built yet, will actually do the exact same thing as the old system. That
That's test-driven So, we did that as the first step. So, this took about 3 weeks. uh where we went through this iteration writing some skills and writing some agents.md, so mostly like uh markdown files, and then running the the AI or the CLI tool against existing code base, and then generating documentation. And we ended up with like really cool diagrams and really cool like, you know, CUJs
and you know, the file gets uploaded, blah blah blah, stuff happens. So, instead of following the path of Qasim, which is the trap in this analogy, where we would just dive into the existing code and rewrite it one code at a time, and then eventually get lost in that particular code base, we followed the path of Morgiana, which exploring the the code path one path at a
time, one potential output at a time. And then rewriting or writing test-driven development to verify that the new code, which is not written yet, will adhere to what the existing code So, that was the loop. So, the the the uh agent proposes things, we run some tests, we catch some errors, we automate the feedback, and then we we repeat. And then we started the implementation. So, the
implementation was, "Okay, here is the code base. Here is the COJs and the documentation. Now, write code that adheres to these particular COJs and that will uh make those existing tests work." One error actually I we caught very early, and this is just learning experiences. There's a couple of errors we caught. One of the most important errors we caught is most AI, what they do is that
if you give them tests to write to to adhere to, and they will have to write code that will pass the tests, what they will do sometimes is that they will just decide, "Oh, the test doesn't pass, so I'm just going to delete the test." I'm going to I'm going to ask you to to like bear with me for a second here. I know it's 9:00 a.m.
in the morning, so I know you're all caffeinated. If a test fails, if you delete the test, it will not fail. That is a valid solution. It's not what you want, but it's a valid implementation, right? Same thing for a database. Same thing for anything, right? From the point of view of an AI, just deleting the problem is part of fixing the problem. You know? That's the
equivalent of actually when you move. So, imagine you move from one place to another. You have all these boxes. I just moved like 3 years ago and I still have nightmares from that move. I have all these boxes that you are not going to open at some point. So, what do you do with them? You put them in a closet. Because they're away of you. You don't
see them. So, they're not a problem anymore. You don't think about them, So, so we had to actually go through this loop of writing code, making sure the test pass, making sure the AI actually doesn't delete the test. And if you're wondering how we did that, it's through hooks. You can basically any agency AI tool today, you can write hooks and in the hooks you can prevent
it from doing things. So, skills and agency MD sometimes is not enough. The rules are sometimes not enough. So, you can actually write code to to prevent the agent from doing certain things. And one of the things you can prevent it from doing is deleting existing files, right? Deleting tests. So, that's one thing. Um then we What we did the next step is basically ask the AI
to actually generate three versions of the same One was in Java, one was in Python, and one was in Go. Because the AI is so good at generating code super fast, what we wanted to do is being able to test or to prototype. So, generating three different languages give us the possibility to actually test every language on cloud and see which one actually makes more sense, which
one is faster. And I'm going to let you guess which one was faster at the end of this. I'll tell you I'll give you the answer at the end. The other thing we did also is that we actually noticed that the system was primarily an orchestration system. So, the ingestion is a file, an audio file, let's say, that gets through this entire long pipeline which has a
chokehold or a bottleneck. And the bottleneck was the monolith application. Which was slow, has to be restarted, blah blah blah. But then when we really looked at how the COJs were generated, we realized that it was just an orchestration system. An orchestration system is any system that basically takes a bunch of files as an input and then passes them through a bunch of business logic steps to
generate an output. Now, some of you might say, "Ah, well, the extraction system, we can just use any other tool to orchestrate uh like a orchestrate code, right?" Um Argo CD workflows, uh whatever tool you want to use for orchestration, that's the only example that comes in mind. But, the problem is because the existing developers had to maintain the system, they didn't really had an opportunity to
stop and think about the entire logic end to end. They just had like a monolith application that just works. So, we very quickly realized what we could do actually is we could decouple this monolith application. Now, again, one of some of you might say, "Oh, this is just an example a typical example of monolith to microservices um re-architecting, right?" Um duh, you know, in flash news, dude,
what the [ __ ] are you talking about? Actually, a lot of times when you are deep into the weeds of the cave, you don't really have time to think about how your stuff right? And this system, among everything this company does, is by definition a microservice. Because the company doesn't only have this, they have multiple other codebases doing all sorts of things. They have front-end applications, they have
back-end applications, they have databases. So, from their perspective, they just see this as one component in a very large diagram of existing components doing all sorts of things. So, they don't really think about it as an orchestration system, they just think, "Oh, it's just a an app that does a specific task." I know like I I did this talk uh to a smaller group before as a
at the test run to this. And uh somebody was like, "Yeah, yeah, but like anybody could think, you know, rewriting an orchestration system uh should be easy. You just like convert it into a microservice application." I will give you a counter argument. 3 years ago, some people at Amazon actually published an article where the title of the article was something around the lines of like migrating from
microservices to monoliths saved us 90% of the cost. So, they had a application that basically ingested Amazon Prime videos, the live stream, and then they did some quality analysis on those videos. And the the existing system was written in microservices, which basically ingest, you know, videos using AWS existing systems, SQS, lambda, blah blah blah. And then they realized that they actually taking this and converting it into
a monolithic application, running it on the virtual machine, they could save costs. the argument of converting something from monolithic to microservice or microservice to monolithic most of the time is a fanatical opinionated argument that people are either arguing for one system as like the thing to do or the other one as the thing to do, but it doesn't matter because everything everything is important in context, and
the context is what are you trying to achieve. In this particular example, moving from monolithic to microservice made sense, but it could have been that a different system or a different application or different use case, a monolithic actually makes more sense. So, um converting this to a microservice architecture where we have ingestion, uh inject like we have a injection pipeline, we have a pub sub sitting in
between, so like a publisher subscriber system, and then we have an orchestrator, and in the orchestration layer, we used actually combination of code and cloud systems. On Google Cloud, you have this thing called workflows. It could be any sort of Airflow Apache Airflow equivalent system that you have or Argo CD or whatever or Argo Workflow, sorry, which would allow you to actually parallelize the processing. So, we
could take instead of taking one file, trying to process it one file at a time, we could take one file and just submitted to a job and then get the job to process that file. Again, this is something we couldn't actually figure out from looking at the application from outside. We had to dive dive in using AI tool to document what this thing does. So, the result
was total of almost 8 weeks iteration. Um 4 weeks plus, you know, homework. for the same existing book that took 80 minutes and cost at $80 with our new implementation in which programming language? I'm going to say and this you just shout a programming language from the three Java, Python, Go. In All of them? Okay, I didn't hear the others. I'll get straight again. In No, Java
was actually the slowest. >> [laughter] >> So, in Go it took actually 2 minutes and cost at $0.28 to process. Which means suddenly we could process even more files when we enter a new market, right? So, one of the things I learned from this particular project is watching the because the project was us from Google plus people from the company like developers in this company together for
a week hacking together code, you know, using the AI tool to generate the code validating it blah I noticed kind of two profiles of developers when it comes into AI. There is the developer type that follows AI suggestions as we all follow terms of cells things which is scroll scroll scroll scroll accept and continue, right? Whatever the AI said. It doesn't matter how you're using it. Just
like accept. You want me to change 200 lines of code? Yes, please do that. Whatever, right? And then there is the other ones or the other profile which I fit in more which is you see what the AI is suggesting to do and then you either accept it, redirect the AI or sometimes even ignore it and implement it yourself. Cuz actually a lot of times you are
much faster at implementing things than you spending time writing a prompt and waiting for the AI to think and then suggestion a a fix or an implementation. And so um if you have ever if you are familiar with this there is this thing called the seven stages of grief um when you have somebody in your family passing out, you know? As an industry when AI came, I
think we have all faced this seven stages of grief. I call that the seven stages of agentic AI. And the seven stages of agentic AI is like shock, anger, blah blah blah, and then at the end there's acceptance. So, I'm going to walk you through these stages. Stage number one, using AI as a search Ignore this is a Gemini, it can be whatever, right? Um using AI
as a search tool. So, instead of using Google or using Stack Overflow, you are literally just Googling things using an NLM. And then taking whatever the NLM tell you and uh using that. This feels like ages ago. This was 3 years ago, 2023. Then the next step is when the CLI started popping up, Claude Code, Codex, Gemini CLI, Anti-gravity CLI, whatever. Uh using it with manual intervention.
So, you would prompt, you would wait for it to plan, you would probably review the plan, see what the code changes look like, and then accept it or reject it, right? Stage number three is you would still trust what the AI does, but put in place guardrails. That's around the time when agents.md, skills.md started popping up, where you would try to direct the AI to do certain
things a certain way. Uh like give it some rules, give it some instructions, instruct it to be a front-end developer, a back-end developer. um how do you say? As a wanna-be Java developer, cuz I like to hang out in the Java circle quite a lot. And uh a very, very bad front-end developer. This is actually my favorite stage. Um I just tell it like, "Hey, do this
this way. I don't really care what you're doing. I don't really like to do front-end code. I don't like to TypeScript. I don't like JavaScript. Don't like CSS HTML. Just like I trust you, whatever. Just do it. As long as it looks beautiful, it's fine by me. It might not look the best. But the point is you are like guiding the AI in doing certain things. Stage
number two. stage number four, sorry. Is when you primarily start with your AI as a primary tool. So, you start there first. you kind of orchestrate it. You you you switch to like you you skip the editor or maybe you use an AI editor like cursor, but you use the AI or the prompt chat as your primary interface. And then code becomes a second thing, right? You
start first with the prompts and then you look at the code second. Stage number five, if you are super cool and I should have had my sunglasses on. That's what I call yellow mode. Yellow mode is you start the tool and you just go like approve everything. Do not ask me any questions, just whatever. So, I actually in this particular project, I noticed that a lot of
people default to this yellow mode. Yellow mode could be don't ask me for for approval or it could be just start the tool in yellow mode and just like implement it in yellow mode. So, this actually was an interesting experience in this particular project because what we ended up what we ended up with is one of the reasons the Java implementation was actually very slow is that
the person who was doing the Java implementation was using the AI in As in he was not actually checking anything. He was just letting the AI generate whatever and just accepting So, it I like I'm sorry for the Java developers in the room. Java is a very verbose language. It generates a lot of lines of code. It lacks a lot of lines of code. It's completely fine,
but in yellow mode it actually generates even more lines of code. And the reason why it was slow is not because the code was bad or Java is bad is because the code that was generated was not optimized. Stage number five is when you become the orchestrator. That's Have you seen this meme on the on social media where people take a picture of them holding a laptop
open and walking with it to the break and say like oh my agent is working so I can't close the laptop. You've seen this before? That's essentially the orchestrator, right? You just like fire up your CLI, maybe three or four of them. You get each to work on one Jira ticket. And then you keep your laptop open 24/7. And then you hope that by the time you
come back from your bathroom break, one of your bugs have been fixed, Then, stage number seven, the last one. That's the architect. That's when you feel you are the master of the universe. That's actually when you don't even use an AI as your chat interface. As one of the CTOs of Spotify have attested on a podcast recently, "One of my Java Most of Spotify developers wake up
in the morning, they type a prompt into Slack, and by the time they get to the office, they have a pull request ready to review." I mean, it's wishy-washy. It's the thing that we want to live. Sounds to me like 2060 or 2070. Like 100 like couple of years from now couple of tens of years from now. But, I don't think we are there yet. I don't
think we are in a mode where we can actually completely trust what an AI can do. I think we still have to verify as developers what it's doing. We still have to be the orchestrator. I think at this point, we're probably not even at this stage. Because as I said earlier, one of the key things we learned, or I learned personally from from this project and from
using AI in general, is a lot of times, you still have to um to to to guide the AI, to like kind of orchestrate it, to watch what it's doing, to validate every single step. And somebody asked me this 2 weeks ago. I was in a conference, and I got this this this question, which I I think is going to be the most important question going forward
for most developers, is how do you shift focus? Right? Let's assume that you have three different terminals working on three different problems. How do you actually focus or shift focus from one to another? So, each time you switch from one problem to another, how do you context switch, right? Not Not in the context of AI, what's context for for LLMs is, but you as a human. How
do you switch from one context to another? And we already know that actually a lot of developers have a hard time already just switching context between the editor and and and chat, Slack, chat, whatever tool you're using, right? Because a lot of times you get interrupted, people ask you questions, uh something breaking production, you get a Jira ticket, maybe you are holding a pager, you get you
get called because there is a problem production. So, how do you context switch? And getting to this level is going to be even harder because you will be basically context switching most of the time or almost all of the time. I think this is sort of the future, in my opinion, where you become the architect, you become the person that explains the how and why, sorry, explain
the why, not the how. And then you just hope or wish or think or expect the AI to do things, right? Um and then the next level over this, which is stage eight, which doesn't exist in the seven stages of grief, is basically auto approve. So, it's the architect plus auto approve, which is you just prompt something and then you get another AI to confirm what the
first AI does or the you know, use different tools to do different things. Some people already does that. I was talking to Stefan uh Jansen, who is the founder of Devoxx, and he told me that he's already doing this, actually. Um my counterargument to Stefan was like, there is nothing at stake in what you're doing. Cuz he's doing a conference, like what's the worst that can happen?
If the schedule of the the conference is down most of the year, no one cares. We only need it during the five days of the conference, right? If the website is down most of the year, no one cares. So, it really depends on what's at stake, what's the most important thing you're trying to achieve. If you're building a system that displays a bunch of text on a
web page and the web page is unavailable for a month or for a week because the AI did something wrong, fine. But if you are building a system that allows people to book tickets for trains, for for a for a flights, hotels, banking systems, then maybe this is not the thing you want to be doing. You want to actually be more here or rather stage four, which
is you actually prompt the AI, use the AI as your first as your first tool, prompt it, but still verify and then approve every single the conclusion of all of this is that I think we are all going toward the future where instead of starting to chop or to wood chop one Jira ticket at a time, one problem at a time, one bug at a time, where
we we need to start thinking about how can we actually switch from this into using the AI in our advantage. And using it in our advantage does not necessarily means replacing us, but rather using it to maybe verify an existing code base, uh reading existing code base, understanding existing code base, um and rewriting that eventually in some form in one shape or another. Um What I started
my this my my my opening talk by saying that most of developers actually don't get promoted because they are fixing existing stuff. Most people get promoted because they're shipping existing stuff. But now I think we can get promoted because we're fixing existing things because fixing it will take less time generally speaking. So, with that, I hope this was useful. I hope I teach you something today and
thank you so much. >> [applause] >> Gracias. >> Is there time for questions? >> I do have time for questions, yes. >> 5 minutes, so any questions? >> Um, hi. >> Hi. >> Uh, well, my question was I mean a similar situation right now where I'm trying to use AI to well, try to help me understand some legacy code and one of the issues that I found
is mostly the edge cases, especially when they propagate not within the code that AI is reviewing at the moment, but rather how it propagates to another system. Um, so I don't know if that was something that you guys faced or how you solved it. >> Yeah, so the the the way we solved that is by also ingesting into the existing AI the interface of that code. So,
the interface being the API spec, right? So, we ingested that as part of the documentation to tell the AI that basically whatever you produce has to end up adhering to this interface as much as possible. there are some edge cases. That would be like an input has a space or some sort of escape characters or things like There isn't really a magic fix for now. The only
way you could do it is you have to write write like super uh, strict tests to to to fetch those edge cases. So, the the beauty of all of this is that actually you could use AI to come up with the potential edge cases. So, you could generate them. You don't have to use them, but you could generate them at least and then look at them and
then review if they actually make sense for your particular system or not, right? Uh, but there isn't really a So, we we had this. Like one of the things we had was um uh one of the problems we had with this is that most my most uh AI coding tools are written assuming people speak English, but that's not always the case, right? So, a lot of the
audio books titles, they're not always in English. Sometimes they're in Arabic, sometimes they're in French, sometimes they're in Chinese, whatever. So, we had to write kind of lots of edge cases to verify those particular systems and because when you write Arabic or you write French for example or you write English, it's not, you know, straight to left, left to right, whatever. So, you have to to So,
we had to like look at this at these cases um one case at a And also, we had to test with as many uh variations of the edge cases as So, whatever makes sense for you as edge case, um what I would what I would do in your case is I would start with those and write those for those first. So, start with the test for the
edge cases first and then move on to the rest. Make sense? >> Yeah. >> All right. Thank you. Thank you. Any other questions? >> Yeah. Can Can you hear me? >> I don't see you. >> I'm here. >> Ah, okay. I see you now. Ah, hey. >> Hey. >> Hey, man. >> Hey. So, um okay, I'm going to ask a question about >> Us. >> So, um
let's suppose that we have a uh a setup where the AI is allowed to retry, let's say, 15 times, which I think is pretty standard. A lot of people are doing that. Um I've seen cases where the token and compute cost for a CI run that fails and times out after 15 goes is $125. >> Yes. >> And that is at current price. Um I don't know
if you saw, but um the uh OpenAI's financial civil leaked. Uh was it Anthropic's? Anyway, they their losses have widened eightfold since last year. So, that means that they are currently underpricing the tokens by a factor of eight. points a a price point for a failed CI run of $1,000 per run in the worst case. Obviously, these are worst case numbers. That doesn't sound great. >> Yes,
I agree with you. >> So, what if this is really the way that we're our our our craft is going to go, how are we going to pay for it? I I don't see my management paying for CI runs that cost $1,000 or even $1,000 a >> Very good question. Uh I love the question. So, there are two answers to this particular question. Obviously, the first answer
is that I agree with you in the sense that eventually, at some point, AI will start costing even more than developers. Because the key difference between an AI as a developer is that AI has universal cost. It doesn't matter where it is. But developers are paid differently depending on which country they live in. Right? Um like a developer based in California will cost more than a developer
based in India. This is the reality. If I remove my kind of like Google hats and I put in my fan of large language models kind of in production hats, I think that's part of the future is that actually companies will start running their own large language models. So, I don't know if you have looked at Gemini 2.5 um or some of the Gemini stuff that Google
released recently. They're actually pretty good. And one of the things that's actually um I think one of the things that I'm missing, there is a key component to this puzzle that is missing yet, is a lot of times when you're using a tool to do coding, right? It doesn't matter what the tool is. It might have auto switch, which is it changes different it uses different LLMs
based on the task that you give it, I In my experience, I noticed that like 90% of the time it just will use the most general, most expensive large language model. So, in my case, I'm using antigravity CLI, I have access to Gemini 3.5 Pro, 3.5 3.5 Sorry, 3.5 Flash, 3.1 Pro, and 3.1 Flash. The 3.1s cost less than 3.5, but when I look at the metering,
it actually uses 3.5 most of the time, even if my question is just look up something in documentation, which clearly you don't need a 3.5 kind of model to do. Right? So, one of the key component that is missing yet is actually a proxy running on your browser on on your editor that can read the prompts, and based on the complexity of the prompt can decide which
is the most optimal large language model to to call. And that that doesn't exist today, right? That's one one thing. The second thing is that clearly you don't need a very large capable model to do most tasks. You need it to do the most complex tasks. You need a capable model. But most tasks can be achieved by a a model that runs on your laptop, So, I
think the future would be two things. Either we get to a point where they run in hybrid mode, which is you have on-device inference, so you have a model running on your laptop that doesn't slow it down, but it still runs perfectly. And then you have a component that can automatically switch between it and cloud. That's one one possibility. Or the second is you still have this
model running on your laptop, and then your company has its own models running on its own hardware. Because by running on on hardware, you can achieve two things. You have predictable uh TCO, test uh total cost of ownership, right? You you you you just buy servers, buy a GPU, run a model, and that's it. You don't have to pay oncurring ongoing costs. That's one. And then the
second thing is you have privacy, also. You can't You don't trust that your your your your editor or your CLI will upload your code into the cloud to a vendor that you don't trust. Like I don't know if you saw recently, but Anthropic, when they released Fable 5 and Metos 5, they basically in their term of sales, they said, "No, no, we use your data to train
the models, So, the future is this. It's a hybrid mode, which is you run between something and either you as a developer, you have to manually switch, or you have a smart component running on your editor, on your CLI that will switch between, you know, on device and then cloud. And then now that I'm talking about this, there is one more thing I'm actually thinking about. In
this particular experience I had, um we need to cut it out. I'm just going to finish this in this case. In in this experience I had, I noticed something. A lot of times when people are actually talking to an AI, they don't specify which file they want AI to look at. So, a lot of them they allow you to do like at like at and target a
particular file. They don't do it. They just say like fix this problem. And then the AI has to spend thousands of tokens looking through the existing code base to actually find a particular bug. So, that's of course also could be an optimization layer, which is you target a very specific file. Makes sense? All right. Thank you so much. Thanks. >> Thank you.
