About this talk
In this talk, Andre, a software engineer at Dynatrace, discusses the integration of abstract syntax trees (AST) with artificial intelligence (AI) for code analysis and optimization. He explains that AI often treats code as plain text, which leads to inappropriate and generic suggestions for code improvements. By using AST, developers can analyze the structure and logic of code more effectively. Andre highlights the importance of AST in identifying problems within TypeScript code, demonstrating how it can provide precise locations of issues and facilitate efficient refactoring. He contrasts the performance and cost-effectiveness of using AST versus traditional AI approaches when conducting code reviews. The session emphasizes the need for engineers to understand the underlying structures of their code rather than relying solely on AI-generated solutions.
Full transcript
You ready? Hope you're ready. Okay. >> [sighs and gasps] >> This is my third talk in this conference. I feel it like the final boss in the game. I'm really feel exhausted. But I hope I will manage it. It's really hard. I will try to make it soft as possible to not bring you in some really dictional technological stuff. So, if you're ready, run. So, let me
introduce myself. My name is Andre. I'm a software engineer in Dynatrace. And Dynatrace is the one of the biggest observability and monitoring company. And as an engineer in this company, I have professional damage. So, I'm measuring everything. And as you see, I even measuring my own presentation. So, I cannot live without monitoring everything. As a person, I also writing books about Next.js. So, this is why I'm
so curious. And how I will end it in this talk. Because nowadays, we have LLMs and huge models that is treating the code as the text. And for me, it was really really I was really wondering why so. So, if we cannot use something that we used before, and I tried to use abstract syntax tree to combine it with the AI. AI fails not because it's incorrect.
AI fails because it's just reading the text, and sometimes it's not understand what exactly is on your screen. So, we also have Copilot, Cursor, ChatGPT, whatever. And if you ask like optimize this component, optimize this code, it start reading this code as the And this is the whole problem. Because the code is not the text. The code is the logic. It's the code. what does it mean?
We are lacking of something that could be exchanged with some technology that we used before like for decades actually. So, AI cannot understand what does it mean, can generate suggestions, and it work with your code as a text. And yeah, it can see patterns, of course. in reality, most of the time if you're using the React for example, it will just put the use memo everywhere as
optimization and it's like, what? What can we do with AST? So, we can read the code as a code, as a logic. And in this structure, we can understand the code. We can navigate the code. We can work with the syntax tree, of course. And we can see exact locations where something is located. We don't need to ask AI find me something. We can just use the
regular logic that working before for decades. just short introduction what does it mean. So, abstract syntax tree will take your sentence like this, like absolutely simple constant X is A plus B, but it will see it like variable declaration. Inside of this declaration, you will have binary expression with one identifier and another identifier. So, it is logical already. It's not the So, we can use anything that
is on the market right now. So, I love Babel. Babel is really cool. It's fast. So, you can use one. You can use cheat code shift for this. Your decision. what we can do with AST? We can precisely navigate through your code. So, we can read it, navigate through it. We can transform the code with AST. That means that we not put something on line We took
exact expression and exchange this expression inside of the tree. And then we can analyze the code. I can give you a hint. This is not something new that I brought to you. Sonar Cube is using this for decades. So, they're having the same logic to find the code smells and the errors in your code. And they're doing the same thing like abstract syntax tree will help you
to find where to do something. AI can help you to answer what to do because now it's helpful. Now it will be concrete prompt that you're bringing to the AI. So, we can find the location AST. But, what does it mean? Like, okay, we found what to do. Fix it manually? Okay. Solution. If we leave AI alone, it will bring you a lot of analysis and will
bring you a lot of generic advices. the biggest problem is assumptions that it will bring you. Bless you. If you combine both the technologies, that will give you a concrete tool that will help work with TypeScript. before I start showing you the example, the thing is that current models are written in Python mostly and how I get into this assumption I just seen in the logs that
we're using traps and we're using Python functions when we're working with the typescript and for me it was like why? I'm not using Python here. It's typescript. Let's use the typescript for typescript, right? So, I created example that is absolutely broken. So, working bad, it's having tons of anti-patterns and my idea was to find the difference. How AI will find this and how AST will find this.
working with a syntax tree doesn't mean that you just wrap the code and that's So, you will need to introduce some manual logic inside of your code that will find reports like this, but you don't need AI to find reports like this. So, it's just one short example that on one line, on exact place it found that we need to stabilize handler. That's it. It's enough. It's
already a prompt. We don't need to read the whole code by AI. And in the smaller components it's fine, but just imagine that it's like huge code base with uh 30,000 online support and a lot of components that you just do not know what what exactly is going on there. And to fix something, you will just need the whole bring the whole context to the AI, read
it and to get only one thing. Just generic use memos. I compared it. I gave it uh the full context, so the huge component uh to AI. I used um Cloud for this. And I spent 1,000 and and with a half tokens, so like not much, but a lot. If I will use more components, the number will increase. And using the AST, I found exact places where
the problems are located in the code, and I spent 260 tokens. This is way more better. So, we can now speak about the working with the code reviews and code smells in our code and not spending tons of money on it. this is the result of correctness. Both of them are correct, so I cannot complain that AI made a bad job. But, the thing is that using
the AST, the strategy was better because it it showed me exact what I should do. And with the full context, it just rewrite the whole code and just showed me like use use memo here and here and here and here. Generic. Don't want to do this is the next thing, and it's also important, so it's already what I mentioned is the efficiency of the tokens. So, less
tokens with better result for me as an engineer, it's way more better solution than just throw everything into the AI and just pray, "Okay, maybe it will be fine." less information produces better result. This is already a quality. So, how I did this? With the pure engineering. So, I just created the several scripts that is connecting to the API of cloud, send the data there, and just
compare the result. The trick is that it's was not like one experiment, just try it once, and then okay, bring it to to public like, "Hey, I you see it's better." No. several hours on this. It was about 800 of experience with the different and the numbers was pretty close to what I showed. the API migrations, it's one thing that LLM cannot just even guess. it does
not know how many size do you have. It does not know how many components you have. It just need to read everything through and bring you the result. Hm? With the AST trees, you can just run one script, get all the information, spend less than zero tokens on this, and get the strategy how to fix your code immediately. this is the just comparison of how the way
of AST trees and how the way of AI insertions is working. I just found exact line, exact function where should that should be uh intercepted, and AI just after insert everything, like without even controlling. So, we cannot even know what exactly is going on. And And as I said, on the smaller components it's but for the bigger components, hm, we can talk. in the next step, just
uh test for this to understand if LLM will work exactly as I want. If it will be alone. yeah. test itself is on the screen, so it's uh simplified, but it's exactly what what I used. >> [gasps] >> can we do with using of AST approach? So, we can use it in the code reviews. Your code reviews for the TypeScript, so just listen to this. I'm not
saying that for any language, only for the TypeScript, it will be way more better. Refactoring will be way more better syntax tree will tell you what exactly are you doing. You can easily find the pattern. You don't need to ask AI find the pattern inside your code. You can just provide the syntax tree and say, "I see the pattern here. Find it." And with the report from
AST, AI will work better because it will be more precise prompt. And for the generation is like perfect thing because AST reads the code as the code. And it's not the text. And then to make you a gist of the commentation from um AST is way more easier because you already see the structure of your functions. So, what can we do with it? So, we can create
the whole flow. AST will analyze everything. Then we can create the prompt for AI. Then LLM will generate the result. Then we can test if correctness is fitting your uh expectations. And then we can apply the PR if it fits. tooling that you can use and I used for is on the screen. You can use Babel parser. It's for me, as I said, it's perfect one. Uh
code API, TS mod, JS code shift. And you can also use something to after apply everything. So, I created my own script. If your company's having something else, your choice. What you can do at next Monday. You can try it yourself, actually. So, I'm not telling you that I'm 100% sure because everything is measured. Everything can be not true. You can try it yourself. So, pick one
task. Use the AST to create the tree from your function. Then, try the both ways. Ask AI to fix it, and then ask AI to fix it using the information from the tree. three final uh things that I want to tell you. The AST is really, really impressive thing, and it's existed on the market for decades. It's where I started. Do not close your eyes on it.
It is there. Use it, please. The context that we're using for the LLMs should be precise. And it should be precise not only with the properties that you're using. So, in my previous talk, I said that you can use YAMLs, for example, to precise your your uh prompts. In the code reviews, you can use AST It's way more uh efficient than just writing the text with a
tons of water inside. And one important thing that you have to do, don't be lazy. Please. It's your decisions. It's not AI decisions. Write your code code No one with will exchange you ever. Trust me. Thank you. >> [applause] >> Questions. Shoot me with your questions, please. >> Can I raise my hand? Hi. Uh thank you. Um my question is basically about the AST and how you
keep the AST up to date. Obviously, your code base can grow quite a lot. And being able to keep your AST up to date is actually what's going to assist in the AI generation part. >> So, just just to short explanation, AST and the code is the same thing. You don't need to synchronize it. So, the tree that you're getting from the code, you're getting from the
code. So, you're taking your last code. You're say read it as as a syntax tree. Done. So, you don't need to synchronize it. It's not something that you're um storing somewhere. You're having this in the real time. >> So, this needs to be dynamically generated each time? >> Okay, yes, sure. But, you're not spending tokens. That's the important >> Okay. I I wonder how that works for
kind of larger code bases as well. >> Just try it yourself. I mean, so it it's where works well. It depends on your machine. So, if you have even Mike like me, tiny thing uh can work with the huge um code bases because it's like it's same code that you're running every day. So, it's it the the the CPU is enough. So, just run, boom, you have
the AST. >> For questions, just raise their hands. >> Hi, thank you for the presentation. I was just wondering, so you're just feeding um the AI the AST output. The complete output. And then you ask your you do your >> yep. So, the conversion was like this. I took the the the pure text of the component or I took the AST of the component. And the quality
of the answer with AST was way more better. >> Thank you. >> Hi. Um have you also considered in wrapping this in an MCP server to have it available as a skill to the AI agent automatically? >> Please do it. >> [laughter] >> But that would be the next step, right? >> Yeah, yeah, you can do this. So, it's like it's a regular code that you can
do in your daily work. >> Instead of doing some analysis on the AST, do you think in the future we might models that are built on like the learning is on an abstract syntax tree? Like if you, you know, go through all of open source, pass everything pass everything as ASTs and put that into models. Do you think that might work? >> Good one. Maybe yes. I
would take a look on it. So, it's it's interesting. So, have no idea. Maybe yes, maybe no. It depends on what exactly where we will use in future because currently TypeScript is not the most popular. Python is most popular, Have no idea. >> More questions? >> Hi, thanks for the presentation. Um my question is uh you mentioned that uh the what by a AST we can also
change the code or refactor the code. And um is it doing it like how AI does? Like uh changing the the file mainly or by using the AST like adding things directly into the AST? >> Second option. So, it's taking the tree and it sees where exactly the function is located and it's exchanging the function inside of the tree and then you're producing the code back. I
think lady in white was first. >> If you've never used AST before, where would you start? Like if it if today is the first day the first time you hear hear of it, where do you start with >> In Google. Your your first step will be in Google. Google it. So, how to use AST because it's not uh something that is invented today. As I said, it's
we used it's from the first day when we used TypeScript. So, actually the syntax tree it's uh the mechanism of how the TypeScript compiler is working. So, there is no TypeScript in reality. We having the some kind of code, then it transpile to the syntax tree and then TypeScript compiler bring it everything to the JavaScript. This is how it's working. >> built in your in TypeScript already?
>> Yep. >> Okay, I have a really random question because I'm a UI software engineer and I work in finance. AST is taking it a little bit further. How do you feel UIs actually change in terms of functionality? Could you see an AST being generated for UI functionality that the user is actually now in control of their own exploratory workflows? Do we think about UIs now being
more dynamic and allowing what the user wants to do? Let's say you're creating a custom trade dashboard and it's no longer static, you know, in terms of like layouts and other things. Maybe there's a world in which we have an AST which is actually powered by the users uh in the sense that we now generate UI functionality based on that AST, which is more kind of dynamic
exploratory exploratory workflow for UIs in general. I know it's it's it's a little bit out >> [laughter and gasps] >> but >> Okay, so let's just start from the beginning. AST is not invention. Everyday when you're opening your React application, it's it's already inside because you're working with the TypeScript and TypeScript is not the language, like not the real one. JavaScript is the real one. ECMAScript is
the real one to be more precise. this tree is inside of your building mechanism. So, it's inside of the compiler. So, if you want to use it as something dynamically, you can use Nobody can stops you because you see the whole your React component. So, if you will use the TSX, AST will show you how the this React components is looks like under the hood. And you
can say like, okay, this button is a function. Let's exchange it to another one. That's it. limitation in this. So, you the freedom is yours. It's the tool that you can start using, try. If you have assumptions, just try >> You mentioned that there are several different tools for generating ASTs, and you prefer Babel over the other ones. Can you explain why you like Babel? >> Speed.
So, the speed is the most efficient thing. So, JS CodeShift is written on JavaScript. Uh the Babel partly written on the Rust. Rust rules. No complaints here. So, the fast SBCs also the good one, but as I work with the Babel for years, like Babel is my choice. But speed is really important thing, and it's already was mentioned today. So, there was a question if we have
like a um code base, the the Babel will read it just faster. like my recommendation here. >> Yeah. Thanks for presentation. How this approach with AST is different to language server protocol LSP for TypeScript as well? For example. >> What? >> Uh LSP in code you can add a plugin that also can read the code and suggest you with everything and in a fast way. Is it
there comparable versions? >> That's the good one. I cannot answer you in 100% because something that I never tested, I cannot answer. Yeah. So, I I can try it today, tomorrow, and I just bring you some results, but my assumption, if it's reading the code as a text, the trees will always win. >> Yeah, so I have a question. The AST is basically just a syntax tree
of the of of the code. So, the analyzing part of what you make of that tree, that is something I would say is completely different. How does Like, why would you uh what What tools are supporting the analyzing like the analyzing of the syntax tree? Cuz if you just have the syntax tree, it doesn't tell you anything about what you should do, what you can do. I
mean, what you can do it's it's baked into the tree, but what tools would you recommend to do the actual analyzing of these >> I've written everything myself, so it's What I did, actually, I partly copied the mechanism that SonarQube is using to find the vulnerabilities, code smells, and >> It's basically linting based on AST. >> Yeah, yeah. So, nothing special. So, it's it's the thing that
we had for decades is already Anything? Nothing? Then, thank you very much. You are a man Best team.
