Feyikemi Ogunsanya - AI Agent Fundamentals for JavaScript Developers

This session introduces the fundamentals of AI agents, specifically targeting JavaScript developers interested in leveraging AI within their projects. The speaker highlights the importance and integration of AI tools in modern software development, discussing concepts such as large language models (LLMs), agent capabilities, and the agent loop process. Practical examples illustrate how AI agents can execute tasks autonomously, interact with APIs, and handle user prompts effectively. The talk emphasizes security considerations in deploying AI agents and the need for human oversight to prevent misuse. Overall, this talk provides valuable insights for developers looking to incorporate AI technologies into their applications.

would um like us to you know introduce ourselves if you don't mind go um one at a time to introduce yourselves and what you do while we wait for other people to join the call. Thank you. So I'm seeing uh okay where do we start from? Who wants to start? Okay, let me start. My name is Jessica Aoy and I am a co-organizer of CityJS Lagos. Um,

I don't know if it's the first if this is the first time you're hearing of CTGs. Can I see your hands up? Okay. So, just a brief introduction of who we are. CTGS is a JavaScript community. I'm still going to go over this for more people that will join, but that that's like the the summary of it all. And I'm a co-organizer. were three organizers for the

Lagos chapter. It's a worldwide community with three organizers, myself, Kerry, and GIF. And um yeah, we put together conferences and meetups and um we've had a few conferences in the past about two and we had a meet up sometime last year and um yeah we just gather as a community of JavaS um JavaScript developers to share insights and you know knowledge with one another. What I do

I do freelance front-end development and I'm also a content writer. All right. So, who's going to go next? Maybe maybe I should start calling names. Abomi, can you hear me? I can see Abomi on this call. >> Hear me? >> Hello, yes, I can hear you. >> Can you just give us a brief intro about yourself? Okay. Yes. My name is Abomin and um I'm a front

and I with um JavaScript reacts I I jumped on the call so I I don't know the things you're asking of but yes I'm I'm a front end developer and I'm I'm happy to be >> Okay. So, it's just a brief introduction while we wait for more people to join the event. But thank you so much. Okay, who's next? Deborah. >> Hello. >> Hi. Hi, Deborah. >>

Can you hear me? >> Yes. Yes. Yes, I can hear you. So you just give us a brief introduction and then just you know tell us what you hope to gain from this session. >> Um my name is more on how >> okay >> Oh, it's quite faint. I don't know if anybody got that, but you said you work with AI tools. >> Okay. Can you hear

me now? >> Yes, it's much better. Thank you. >> Okay. Sorry. So, I am an AI software developer. So, I I was excited when I actually saw this um event. So what I hope to gain is how to utilize AI tools for more JavaScript futures and and so on. Yeah. >> Thank you. >> Thank you. So Oh, D Bryce. >> Hello. Good evening. >> Yes, I can

hear you. >> Okay. Good evening. My name is David Bryce and I recently started learning backend development. Started with NodeJS, so I'm still working my way to it and I'm happy to be here. >> Thank you so much. Who's next? F. Oh, that's our host. Oh, I see gift on the call. That's um al she's also a co-organizer of CTDS Lagos. I remember when I was talking

about what we do. She is the She's also part of the organizing team. Welcome Guardian Okon. Hello. >> Hi everyone. Good evening. Um my Can you >> My name is Gordian. Um I'm a back engineer at Trackly MG. uh what I'm getting from this call is how to integrate AI better into my workflow. >> All right, that's that's good to me. Okay, since we have um I

think we're about 16 on the call, we can just get started so that we we have a very limited time. Once again, my name is Jessica and I'm your host for today. And um we're really glad to have everyone join us today as we explore a new series um called AI for JavaScript developers. Like I said, um, CTJS, that's for people that are just hearing about us.

CTJS is a community for JavaScript developers and it started from meetups, our meetup in London, and it has just bloomed into this worldwide um community where we host conferences and meetups where developers come together to learn about JavaScript and JavaScript related technologies. So um yeah, this new series we're talking about how AI is shaping JavaScript or the future of JavaScript development. And you know AI is everywhere.

I don't think there's anybody that is on the internet that is not aware has not encountered one type of AI content or the other. Right? If you go on YouTube, you find even big brands using AI for marketing. So it's just everywhere. It's such a powerful technology and we in the space like in this tech space we also need to be on top of what's happening which

is why we put together this series and um today's session is just going to be about the AI fundamentals and what they mean for JavaScript developers especially how you know you can start to leverage the technology into your own projects as well as the work that you do. So um we'll have a presentation by our speaker and after the presentation we'll have a Q&A session at the

end. So um yeah so just a quick one please make sure that your microphones are muted so that we don't have any interruptions and during the session if you have a thought or comment that you would like to share afterwards you can just drop in a comment and then we'll um address it afterwards. Okay. So without further ado, as I say in theater, let's um I'm excited

to introduce our speaker. I'll have to read her, you know, by a bit, but permit me. Um our speaker for today is Fi and Fi is a senior software engineer. I feel like I should emphasize the senior part because you know she has seen a lot and it's not easy. It's not an easy fit to get to that stage. She's a senior software engineer based in the

UK who specializes in building scalable AIdriven systems and developer tools. Our work focuses on areas like AI agents, LLM observability, and modern full stack development using technologies such as React and TypeScript. She she's passionate about sharing practical real world insights on building intelligent systems and enhancing developer workflows. Beyond her work in tech, she's also a dedicated wife and a proud mother of one balancing family life with

a strong passion for innovation. So um please let us welcome her. You can engage with the there's a symbol to yeah please clap your hands. Let's give her a warm welcome. Thank you. Sophie Kami, over to you. How are you doing today? I know today's Friday and then everybody's looking towards you know the weekend already. Um so what we are going to be like learning today is

AI agent fundamentals. Yeah, I know there's a lot of expectations out there about learning and a lot of you know um you know road maps in learning AI but I I also al also listen to when people are introducing themselves earlier about okay they're already currently working with AI and then just to know about more about AI and then integrating AI into daily workflows you know working

like with cursor working with clouds and all but today I I think what we'll be dwelling more about is AI agent fundamentals you know how to even build an agent um so I will start with um firstly um demoing an agent with us like what an agent AI agent is and what LLM is so firstly I'm going to just share my screen. think I'm sharing this. So,

let me know where you can see my then. Keep let me drop this as well. Can you see my screen? >> Yes, we can see your screen. >> Okay. Yeah. I don't know if Karen can help me share the two links I shared earlier so they can have access to the to the notes. Thank you. So currently what we are looking into is you know AI agent

fundamentals we have um intro to LMS and then uh before that let me just you give us what we'll be building. So we have an AI agent and then if you go into the repository you see how the agent is being built and then you will need normal open AI key. So if you don't have access to so basically you need to like um get access to

the key in order to run the the codebase basically. So, so this is an agent, but what the agent does basically is like general purpose um from principle to call loops, memories. So, it can basically search the web, you know, have access to your files, maybe delete um or write to to files basically. So if I come here and then maybe um do MPS start first and

then you might also need why that is um so you'll need an EMV file. So this is where you put your open AI key. Yeah. So so um I have this running now. So this is the AI agent. Um this is the AI agent um is you know the agent is up now. I'll I'll show you how to build it later but just to see what it

takes and all. So if I come in here and said, "Okay, can you um maybe quickly so quickly um finding highest paying role in the board and then and then put it in a markdown markdown table as job md. So first this job md is like very new. this file that's the job nd it's not been created on my folder here. And then when I so it's

basically a chart. So if I click enter so you could see that it's thinking that's like what they call chain of thought. So it's thinking and then it a web search that that means he has to take a particular tools and then now it decided to take like another um tools like call it um another again. So um so now that's been done. So it says do

you mean the highest role among you know everything and everything if there's some search? So I'm just going to say yes. So I guess there's a title in what I type and it's still searching. So at this point um our our agent what what he's basically doing is he's b basically just picking a particular tools and then so now he's asking some you know follow-up questions as

well. Okay, quick questions before I said do you want the absolutely high is ro. So um this is like having feedback before you even um do anything. >> hi P. >> So I'm just going to say yes. K can you >> Yes. Please can you um it's a little faint. Yes. If you can zoom that would be really great. Yeah. What about now? Is that better? >>

Yes. Yes. Thanks. Okay. So, as you can see, he's asking like a lot of follow-up questions like So, um I'm going to ask another questions like um just simple questions like um maybe um um quickly for maybe five recipe. Um, put it in a matter so in a month file So I gave it another tax because the first one was very vague and that's what um prompt

engineering is because first the first the first prompting is very vague like it's very very complex and now this one is kind of narrow the second prompt I'm giving it is quite narrow and then even though with that we can see that it's it's taking a one particular tools that's web search and then it's you know looping in until it gets um a very final and a

very good um recipe and then from there it's going to like um search. So we also see that okay after the search we have two approval required. So, he's asking me for some approval before it does some. So, I'm going to go ahead and put on yes. let's say yes, please. So just bear with me. I'm using a very different um keypad and okay. So now um

you could see that it's created a recipe MD and then I'm just going to drag this and So, it created a recipe file by itself. And um let me just preview table from here. Yeah. So I just preview it so we could see it like um the ingredients of the the of the jalof fries the directions the notes. So um another thing to notice there is I

asked him to do one thing and he's giving me direction notes and all but yeah that's just agent. So now he has access to we could see some approval we could say um not only web search we also have write files tools and we also have delete so if I had it to delete it's going to delete this particular files so these are all the you know

the tools I have here but that's just basically how it works so in order to see how you know how this agent is being built from scratch because you This is cool to see. We know that we have some prompts and then he basically just write to the to the file even create the file itself. So yeah. So let's see how all this all this um is

been build. So I'm going to go to So today we're going to look into you know L&M which is the brain of everything like the brain of how all this is being done we're going to look into agents and then to calling. So in order to like run this project or maybe even look at it and how it runs, you might want to like just like I

said an API key for at least one model LLM provider. So you could go into open AI and then create an account. Um I think you'll need to like spend maybe one or two dollar but yeah that will be that will be it. Okay. Then the other one you might need is lamina for for running and you know and it's very optional but yeah let's start with

what's an LLM you know an intro to LLM. So um you know an LLM basically is like a train model that predicts the next um piece of text like maybe word token or even characters. So um based on everything that basically came before it so when we are saying um lms so let's just break it down. So we have large, we have language and we have models

which what LLM stands for. So with LLM with large we have you know it's just been that the model has been trained on very massive amount of text like you can think of it like having a friends that have read a lot of books like um maybe they the friends like to read a lot of books. he has you know traveled seen seen the world and even

drew a lot of cookies and all. So basically a friends that has a lot of exposure then you can pick knowledge of that friends but the difference with LLM is that you know um it's been training like it's been trained on English like you the language means just normal human language like and even any other language but with charg the majority of those use English but with

um with model it means it's basically trained with um it means instructions. So that means it contains a bunch of name probabilities. So uh it's not magic but it's more like a statistical machine that's been trained on data. So what that means is um with with so because we have we also have small um large models but these times they have the reason why they they called

large language models is because they are trained on a very large parameters or even large data. So what happens during training and how do they learn just like you think of it like you are maybe learning how to do basketball or even play basketball. So you know you throw your body adjust the failures and then the correction and all. So that's just how it works. You know

it makes prediction. If it's wrong, you punish it and if it's right, you basically just reinforce it. So over time, he adjusts its internal probabilities which are also called weights. So like you're choosing different parts in a decision tree then eventually it gets good enough that that's when you want to stop correcting it. And then um now you can now you know you now got a trained

model and then so at the end you basically got a five full of learn probabilities because by the time it's wrong you already say oh this part is wrong. So the next time he knows that oh this part is wrong I'm going to um pass the or take the next decision. So that's how LLM basically work. So there's no magic into them. So they train them on

how to go and then at the end of the day what they do is to predict the next word. So um so because it has to like predict the next thing that comes or maybe the next thing that think of it like an autocomplete but with autocomplete is just um you're not guessing at it goes as maybe as the user types with LLM it's taking all the

words and then it's guessing it at as the user is typing basically so because um it has to be good at predicting the next uh thing. Then it has to be trained on a lot of stuffs. So that's where the LLM's the large also you know comes from. So now let's move to um how it works which is token. So um with tokens LLM does not you

know they don't read um the way we do like you see what the model sees tokens. So to is just a chunks of text. So that could be maybe a full of words, a part of word or even a single character. So before model processing anything like before they process for instance if you go to charg and you type um what is the latest news or maybe

what is this and this they don't they have to break what is the it has to break it down and that's what uh tokenization means you know and by breaking into breaking the word into like maybe reusable pieces the model can understand word that it's never seen before. So talking affects everything that you know that you do like if you're building an AI. It affects the cost,

the speed, you know how much test you can send and receive and you know with everything that's context window. So with the context window, if you go to charb and you put in a lot of text or maybe a lot of paragraph into it, it has to like grab what's important in in it. And then with that he also have to like reply you with text. So

with that with the input and the output that's context wo because he has to remember all of it and then grab what's important in that um in that text in that um maybe sentences or a lot of um paragraph. So yeah so that's how works. So to talking are basically the currency you are spending. So if you're building like AI agents um you have to like keep

this in mind because for each of the input that you send to charge or even if you're building one you are spending cost and you you know it also affects speed you know the amount of um output that LLM have to put in also affect the speed that so as only he has to just generate a smaller token it will be faster than just um if I

if he has to like generate a very very long tokens. So now um so always think of tokens and not when you you know when you're building an AI agent. So let's move to um you know the limitations because obviously we have LM and what they are very good at is just very being you know um proactive whereby whereby um sorry hallucination hallucination means like um when

LLM is lying but not really like lying because when you lie you know that you're lying but with um with LLM they they don't know that they're lying. they think they are actually saying the truth based on the data they have been trained on. So um for them and if you look at the answer they have been generated maybe it's wrong and what they've been trained on

you see that oh it's not that it's lying but it is it doesn't know that it lies because it cannot think right it just think based on this data and based on my probabilities maybe this is right so um that's one of the limitation of um LLM and then another one is like the contest window because you know just like what I mentioned um tokens. He has

a limited um tokens and a limited amount of memory that um words that it can you know it can remember. So that's why sometimes when you are chatting with charges some it can lost um a lot of context because you know it has memory and it has limitation to it and it also have cut off um time because each of these LLM has a learning cut off.

So for instance GBT 3.0 zero. It's been trained on a large amount of time or on a large amount of data. But this data might not be the latest might be maybe till 2023 or maybe 2020, you know, um because they still have to train it before they release it. So that way if you are using it in 2026, um maybe give it another extra um capabilities

like to calling for it to like call the web and then get the latest um get the latest um information. So that's that for um LLM. We also have So with limitations with um LLM we have let me see we can show this to so with agent there's actually no like universal agreed definition with agent it has you know different people has different definitionations to it. So,

but to me I think uh like very very simple practical definition to it would be an agent is is that LLM you know that can decide what action to take using reasoning and tools to complete a tax. Yeah, you see a lot of definition like when you give LLM tools and then with what they call agency and then he has to like keep looping until you know

until he actually finalize a particular you know answer. So it's either maybe he has the answer or maybe he's hallucinating or maybe he calls a a tool and then with the response to it he also like want to think about the the event that is so the part is not with agent the part is not predefined like yes you are still the one that write the code

you want to do with workflow you want to do if a happens then you want to just like normal if else statement in JavaScript you want to like if it happened then this happened but but with agent the part is not like predefined like it does not it's it's on the runtime that determines what path to follows basically so the similar um analogy will be programming concepts

so for instance if you are looping over a simple array because array is fixed Right? So you have maybe you're looping one to 10. You already know I'm stopping when I get to the last um array. So you could use normal loop. But when you are traversing like an unknown tree. So that point you have to use recussion because you don't know how deep the the tree

might be or maybe the the width of the tree. So by that means you want to like u use precaution. So same thing with um agents. So also with driving so normal a normal LLM will be GPS giving you direction. You know an agent will be driving the car itself. So that means the agent is the one you know driving it car itself but LLM is you

are still the one driving but it's just giving you direction. But with agent you let it determine how if I want to pick this particular tool just like the demo that we saw earlier you know I didn't ask him to pick this particular to he pick it because he want to search and then just like um you know just like we saw so with agents you are

not like in control of each steps you know so agent let's also see what agents are good tax because you know you don't want to use agents in every t just like you are not using recussion or maybe a particular decision for if you're using code. So work for workflow normal ways of what you what you should build will be workflow because workflow is like you have

this happened and this does not happen. Then with agents you want to think about is this problem is it open-ended you know if the problem is open-ended you know you don't know what might happens maybe if a customer comes and says I want to like get the refund or maybe um book a trip for me you could get to that website and they said okay the flight

is canled and all. So at that particular time you want agent to be in charge you know the solution another thing is thinking about the solution part changes each time. So you don't have control of the input and the output will stay the same every time. So at that particular time you might want to think about um having to um use agents instead of normal workflow. Um

but if you're thinking about you know what agents are bad at what where you could just use maybe workflows and then um just um normal code and if and else statements will be like physical work tax you know for now yes we have robots you know are coming but it's still like early and then you know there's a lot of robots out there that it's not even

perfect but yeah so we it's with you know tax requiring physical presence and then you know we also have eyes take decisions like thinking self-driving cars so that way you still need human oversight and then we also have creative work you know with creativity they don't truly you know because they are trained on data that already exists so you don't expect them to be creative they what

they can do is just predict and what they predict on is what they've already been train on. So you know also real time and low latency t because improving this is also improving but still you know there's a constraint on depending on how it's been set up and how so um let's talk about agent loop. So uh and this is what makes agents very very autonomous because

you know at the core of every agent is a loop. you know, you receive a tax just like we saw in the demo. And then when it thinks about it, that's reasoning. So, it thinks about what to do. That's also called chain of thoughts. Let me break the tax down into the simplest form. And then from there, it takes an action or maybe response. If if it's

just like um what is so so so it responds straight forward. If it's what he knows, that's if it's what the training is on. And let's not forget that agent is still LLM. But now LLM with tools with a lot of capabilities and also um memory which is the context window. That's what an agent is. So it's still the brain of it and the art of it

which makes use of transformer architecture. So um the agent loops so you receive a tax think about it and then choose a tools. So when you feel like okay I can't do this um on my own if I have to do it I'll have to like give you the wrong answer that's hallucination but okay I I see that I have a tool that can do the existence

that's then you choose the tools or maybe take action then execute the action then when the action call and u maybe use the web search results it kind of observe the result as well it it does not just give view the oh this is what I have from the web page and then throw it no it thinks about the result of the um calling the web page

and all and then you repeat that and that rep that repetition is loop it continues that until like okay this answer that I have is good then I'm going to give the user this particular answer so it's also sometimes called the react pattern you know reason and heart so the model reason about what to do heart and then reason about what it also observes. So yeah, the

future of AI and what we have out there about you know um when you see um LLM all these companies doing competitions and all it's just because you know there's a lot of things to still be done and because a lot of demand is been there for agent a lot of usefulness for it you see um better tools used So a lot um work is going on

into you know longer context because currently I think the latest charge which is 5.3 is making use of a lot of um context where window context but then it's still not very well um not not there yet basically on what um people are people want. So something like multi- aent agent systems whereby you have some specialized agents you know just give um just train a a an

agent on maybe to be a doctor or to be a nurse or even you know just what we say like or maybe cloud code just to specialize on one particular training and then by that not some general um data and then you know so the future of AI if you ask me I'll say you know it's very exciting because a lot of work has been going into

it and then um let's look at you know if you've not maybe called out an agent before um like it's it's what I would say like very very simple actually to build an AI agent is very very very simple but to build a good production AI agent That's you know that's the artwork and you know I don't know if I mentioned you know there's different between an

AI agent and a machine learning engineer because an AI engineer rather and machine learning engineer because you know with machine learning engineer what you what you are specialized to do is to build and train these uh models that's the LLMs you want to do a lot of finetuning you know the probability that we discussed earlier about having to train let them have a very good instructions and

then knowing the part to take when they face the particular decisions that's what an um machine learning engineer would do but with AI engineer you want to take the LLMs and build applications like charg so chargbt is an applications is a chart application running on LLM. So with AI engineers that's what um you will be building as an AI engineer and then so this is like a

basic form of you know it's very basic I'll say this is this is still an LLM because what we are doing is um if you like copy this and then maybe go to I have um so I have um for that same repo you could like check out to the lesson branch. so if you don't get check out And then you just copy this and then throw

this in your so what you need you don't need to like the UI. So what the UI is controlling is the CLI or what you're seeing here just to make it uh looks good. But basically what you want to like maybe out is this agent and then you want to like go to your run and then just paste this code in there. So what we are doing

here is very straightforward. So we are using OpenAI SDK. So without this SDK what you'll be doing is writing a lot of just like you're calling like API calls and then you want to you know you want so you you'll be writing a lot of basically just like you're calling um a normal API but with with open AI it's just like a rapper that allows you to

not write a lot of boiler paint code. So yeah, so what we have here is the generate test is telling us you know we have normal just like you have with chargational um applications and then you know with that we are calling it and then we are passing user message and the conversational history. So that's the previous history so far. And then we have callbacks. So the

callbacks basically is for is for the UI. So don't worry about that. The most important thing here is the user message. And there you can also call it can also pass in temperature. You know with temperature it determine the randomness. So but if you leave it out your agent will still work just like of temperature zero. Um so with temperature you want to control the randomness and

one thing about AI just like I mentioned is that is it is very nondeterministic because um just like normal code code or workflows they are deterministic you you the one you are the one that said if this happened then do this you know you know if I give this function this input it's going to it's going to take all these outputs basically but with agents you don't

know what the output will be like this function that this uh demo that we we saw if I had to put it again it won't it won't the output won't be the same because first one of the um one of the factor is um temperature and then if you have temperature to be zero it will still kind of control the randomness that's the deterministic um then um

another way of testing it is writing a lot of evas you know with evas is like you're writing test but now you're not expecting true or false you're kind of think think about it about like you're writing a snapshot um test where you are being more quantitative about it not um just false or true or false basically so yeah so what we are doing here is just

So doing and then um writing this function you know it's basically writing a function and then doing some asynchronate and then um passing the necessary um parameters. So here I'm passing the user message but by the time I'm running it. So I'm going to just do npm install Okay. So, I have my AI agent running now. And then what I want to do is just to give

it the questions. Okay. Yeah, I already put in the questions. Uh that that will be the user uh message just like you go into and you put hello can you hear me? But you could see that it says I can hear you. Okay. I can't hear audio. Can read your messages. You know how can I help you today? So this is already smart you know you already

detect that okay I can't speak I can't do this but you know how can I help you and what I can do with this this is just like LLM right you know this is basically LM what I can do with this is just um you know if I ask let's ask another question maybe yeah see that's that's the thoughts it's basically um breaking the process that he

has to like okay I have to do this and do this and then do this before I answer these questions okay so I think I have to Make chain this. No, I'm not doing that. [laughter] Okay, let me do So, what is the latest news today? I know we know the answer, right? Okay, that's the latest one AI and this is auto making life very fast right

now. Um, okay. So, I'm just going to quit. I'm going to run it again. Yeah. So, I assume the like the answer will be because just like what we discussed about their training, you know, cut off. Normally it like it does not have access like what we just build. This is this is just what we have. So it should like um maybe give you some direction to

know the latest today. Maybe write you can go on web page and all. Okay. So now we have our answer. So if you want let me try So yeah we we have the answer and then is it's like you know I can't browse the web in real time so I can't fetch today's live headlines. So if you if you tell me a region or topic eg world

us and all. So, so basically it's just telling me quick way to get to this airline because it can't do it. Why? Because it does not have access. So um and that's where you know two calling comes in. That's um basically function calling you know. Um but before we go into to calling and all basically this is like LLM without tools without l without memory you know

what we have here is just um it's an agent but it's just an agent that we are building on ourselves and then um so that would be so let's go to to calling by you know just give It's think I just so um two calling is simple if you ever write a function that does that have inputs and then outputs you know it's also sometimes called funure

calling you know it's same thing but you know how to you know how to calling works you know you divide the two you know you want to because you are the one developing an agent now so remember we already talked about we giving it the tube how do you know you have to like line it and then we're going to look about look at it later just

like this you have to divide the two just like a function you know get day time think I have get day We just declaring just like a function. This is JavaScript but it's also um this TypeScript but it's also you know if you're not familiar with TypeScript is just basically JavaScript so you can forget about the TypeScript for readability process. So to call what we want to

do when we are developing an agent is that you want to like define each tools we have a name you should have a description and parameter and then the the description and the name we allow the agents to okay this is what to doing but the mod is So if you come here and then look at these tools for instance you know you get date time the

description is like get the current date and time think I might need to zoom I'm trying to zoom Do I feel like it's move? Okay. So tools what we want to do is we want to write the description, the input schema and the execute. So the execute will be the actual implementation just like so when an agent like what we just saw that okay I need to

do I need to write an um I need to like call a web um maybe use a particular tools maybe write to files or do something it does not just run that function itself because of security reason and that's one of the um that's one of the um skills in building a very good AI agent you do not just want to give it that function that you

just write, right? You want to first give it a very good description. When you want to call these tools, what you just have to do for me is just tell me that you want to call these two. Then I on my server I am going to execute it to you and then give you the input that you think about and then you decide if you want to

respondse to the based on the based on you know based on my input but for security reason you know the function has to run on our server. So you know you it's you know for description it's very critical for model to understand when to use these tools you know you have to be specific and you know majority of it you just have to be one sentence you

know then you want to help the model to decide when to use it basically that's what the description is then this is how to create tools just um you can use z with zord it helps to you know have the parameters on runtime. Um so with index index ts what this is doing is allowing us to like save a lot um tools that we might have. So

here we have get date time but you know if you if you look into the demo that I the code that we have we also have a lot of tools you can have delete files and list files write files read file execute code web search so all of these are functions you know you are just writing a basic functions and giving it and exposing it to the

um to the lm that's what two calling is you are still writing a function but the only difference is you have to have a function that runs these tools and this is what the execute tools is doing. So with the execute tools um feel like my time is up. So with the execute tools what you want to do is you want to dictate when the LLM wants

to run that particular function and then you want to run it on your server not give the LLM oh this is the function go run it that means you're giving open AI to run the code on their server and that might be a lot of you know security um concerns and and then yeah Um I think the key point here is that you know tools are very

declarative. You describe what they do. The model decide when to use them. you still don't decide when okay you the only thing that you want to with your prompt maybe up here just like I showed earlier you can said okay what this tool does you can do you can kind of like say use these tools where you want to maybe search the web so with that you

are also kind of determine the controlling the agent being more deterministic because you kind of know if you call this function if if you ask the LLM to do this particular task it should use this particular tools. So with your prompt you could also control that particular um behavior of the agent. So then it has to be the description has to be very good. So because the

model will be relies on your description to send it the right tools or not and then you know could use schema validation and then you control the executions just like what I mentioned because that's the security boundary. So yeah, I don't know if any anybody has questions. There's still a lot you know to build this you know aside from the executions. We still have you know is

what I know just like I said is you know um it's very easier to build these AI agents but uh I think going forward as an AI agent this is what you'll be doing on a daily basis because now you've built your agent and then you have is just writing tests but with test just um like is very different you want to even write on productions, you

know, get um live data and then test them and then even test your agent behavior. For instance, if you ask if you have a maybe um an agent that helps with cooking, you could test it based on if I give you a mob, you shouldn't go and mob the floor, right? So, you are testing the particular characters. So, and you know just like you test normal code

but now you want to like um do it you don't want to like um have a true or false you want to like um test it on a very quantitative um manner so that way you have some scores and then um um the scorers were based on some behavior as well. So um that's all for today. Sorry to just went to sorry for interrupting. Thank you. >>

Yeah. I don't know if anybody have question. >> Yeah. So please if you have any questions me I've learned a couple of things. You know when you talked about LLM hallucination and you know how it generates information based on predictions you know based on the data that it has been trained on and also about you know autonomous agents how they like the decision makers right. Yeah. So

very very insightful. Thank you so much for sharing. Please if you have any questions, can you can you just uh raise your hands up or you can type it in the chat and then we can go through Thank you. Do you have questions? Questions? Anybody? let's hear it. Hi V. Thank you so much for sharing everything that you have shared so far. I think I'm very very

curious about the evals and all but I know that this like a very short call so you're not able to spend too much time on that. Hopefully whenever you have time maybe we can bring you back and have you share with us more information if you have us. Um but I have a question around um you mentioned something around AI agents normally is fine on your laptop

and all but when it moves to production is a totally different ball game. So I know you have experience like working with AI agents or production and all obser observability. I just wanted to know like what are things to look out for? What are things to keep in mind when you're building for production? Building an AI agent for production. What are the specific things that you've learned

that are things that an AI engineer should always look out for? Yeah. Yes. [clears throat] Yeah. I think uh that's a very good question because you know just like I said it's very simple when you just build an agent. I think the first one is the confidence level. you you just like I mentioned they are very nondeterministic and that's why we love we want to make use

of it >> because they are agents >> but when you go into production you don't know what the user will be putting in your agent what the request will be you can guess and that's what you will help you do you want to write and then have like 80% confidence before you even move to >> But one important um stuff to keep in mind is security. you

know your agents can your you know you don't you can have very very um maybe bad people or maybe bad at all that can give your agents some scripts to run and then that way he exposes a lot of data >> to the bad actors and then that that's hacking your entire system and then you're gone overnight. So that's one thing. So you have to do a

lot of securities and then another thing is token like I said you know token is one thing. So there's a lot of just like normal API calling you have to put some weight um limiting on your API because um and per users because per user request you also spending money. So imagine when your users have a lot of um request and then it you know it cost

you a lot of money. So you for each of the users you also have to put this is token that a users have to use >> you know that way that way um you limit what the users we you know be we call on your server. So another thing that is very very important is you know uh the context window you know not losing a lot of

um good context that's memory and then the way you could use it is maybe you could think of summarizing >> you know when maybe you already think about the the content to use is full >> just summarize the previous one just take the important that um aspect and then put it back. So that way you you still lose all the previous um context but now you've already

summarized them. And then when it comes to testing the um the LLM output as well, you could also use another LLM to judge um the output of your LLM that's majorly used on production and which is very good because at that point you are not always there on the house. So you could use another LM to um you judge the heart of and that way um by

the heart you could you know either get um bad or maybe good and then you give it back to your LM to decide if you want to still go. So which is which has been very very um good in determining on production if you want to um go um give a user a particular you know answer based on your LM. So using another LM to judge what

the the previous LM um actually responds to. Yeah, I think um there also there also um others um point I could say but I think that's like the one I can remember right now. >> Thank you very much. So rate limiting security confidence level context window management and LM I'm taking those. Thank you very much. So the floor is open in case anyone has any other questions.

I do have some just >> Hi, if you can carry on with your >> Okay. Thank you for that session. Um, I just have it's more like a clarification that I need because I didn't really get the context you said about not running your is it functions or methods um on your system or running on cloud. I didn't really get that part. If you can shed more

like can you remember what I'm trying to say? >> Yeah. When I was talking about two how do we so what we are doing um if we remember our demo right by the time LLM let's say we build just like what we demo and then we h it was the latest news for today it doesn't know it. And let's not forget the SDK that we are using

that's basically let's even forget SDK or this thing it's just normally we are calling an API what we are doing is we are calling another server to in open AI so by the time you are calling an another server in an open AI and it's saying I can't do this and um as an AI engineer the two calling what we are doing the two calling you are

the one building it and two calling is like that's the point where your business logic actually um that that's where your business logic actually meets LLM because at that point you want to write a lot of function for what we demo we have read file write file a lot of functions so all this function that's what the LM will call and that's what the two calling is

so all this function remember we are the one running it like we are the um having that function. OpenAI does not have them. So what I was talking about is when you want to give the two calling let's say with the web search now to LLM it's very wrong some people do it and that's a very good part of building a very good agent. You just don't

want to wrap all the function maybe as you know if and then just give it to oh when you need this this is this function. Why? Because currently is just basically calling a um a function. So what if is a business logic. So you know it's a business logic. Maybe you have a data a lot of you know even if it's it's a part of how you

solve your personal um users problem. So you don't want to like just give it because by the time you give it you expose that function to LLM. You are just think just think about it as giving open AI access to your internet data. So as a business you don't want to do that. You you just want to run it. the LLM will give you oh I want

to run this it's it will be like a JSON like a normal update and then you know the name of the function that you want to run it's I I show it earlier I think I [clears throat] can so that's what the execute execute tools is doing it's basically an as function it will know the name of the u it's also like a function the LLM will

call the function and it will pass the name of the function that initates and then from there you are the one that will invoke that function. Okay, you need this function. I'm going to invoke it and then by invoking it and let me also let me also mention that it does not just um it also will also um like pass in all the parameters that it needs.

So for instance a function we parameters like right. So if you need if that function needs any maybe parameters it will also pass it there and then by there that's where we on our end we invoke that function and then give the answer to the to do that's where you now continue looking okay is this answer correct do I need to call another maybe tools to maybe

continue this um maybe to now call another tools and then pass in that same result of that previous answer to another function. So in our end what we are trying to do is to first protect our business logic you know don't just give because by the time you are giving that mean you are giving all the function to LLM and then which is you know which is

you just see that tomorrow open can just come oh we have this business idea and that's one of the things that is affecting a lot of startups now you see that they have business ideas and then they maybe give them two weeks you see that Google or even big techs are now implementing their business logic and then they gone in. So that's that's one of the ways

they are actually also extracting data from from startups and all. So so you don't want to you don't want to extract data that's just that's just >> sorry interrupt you. We have we've gone past um the schedule so we'll just quickly wrap up. So please do you have um does anyone have a question last question before we end the session? Anyone? May I have a question regarding

security? Because um especially when you talk about the autonomous nature of um AI agents where they able to they're the decision makers, right? And maybe because I've watched so many sci-fi movies [laughter] and it's concerned about security. Have you like experienced any significant security issues that you think that um we should be aware about? >> Yeah. I think um a lot of LLM going wrong and then

doing a lot of things without even having a bad actors. When I said bad actors, that's like a it's not even a controlling them. It's just them thinking okay maybe I should do this >> and I think the major things that happens is prompting you know when you a very vague complex prompting they don't even know the specific and they are very good when you give them

a goal to achieve okay this is the goal you want to achieve and then >> they open you opened about it I don't care how you achieve this just maybe book me a trip and then they just go activate but maybe at the point of them achieving it they actually can run another function um there's something I didn't talked about about neura there's also what they call

neur network which works like human brain and then from even those people that build it they don't like really really know the deep down on it because they don't you can't know Because it's it's just like probability. >> It's not humans that train them. It's not anybody that train it. It train itself based on >> on wheels and probabilities. So, so from there they it could actually

aside from as asination comes when you know you're still doing the right and wrong questions it could also go and do another action based on okay what I think what he thinks is right. And then with that that's why currently people don't trust finance you know yeah there are still a lot of startups with finance and all but it's still very you know people still have trust

issues with with LLMs and all so but for security still the base of it is not even thinking about bad actors controlling your agent now passing normal user rule normal user input to it and then maybe just um guiding it to do a lot of things that even you you don't know. So imagine a user just acting just putting a function maybe a script into your LLM

and say please run this for me and then be you know it it's alreadying maybe you know sending money to another users and all. So you don't want to do that but at some point it's still based on having a lot of you know humor in the loop where you will get approval before agents can even do a lot of things maybe a very risky things you

get human approval before he goes ahead and do it and then that way you can limit a lot of security things and a a lot of bad things that can happen. >> Yeah. All right. Thank you so much. Our So yeah, if um you can still go through notion if you want to explore what it's in there. Thank you so much F for this valuable session. We're

grateful and thank you for everyone that um joined us today. Thank you so much. Um once again if you have questions you can reach out to FA on her socials. this in the chat and then please stay connected with CTJS on IG, Twitter for more um info about sessions like this. So thank you so much for your time. Thank you for staying tuned. We hope to see

everyone again sometime soon. So check the chat for we also have a WhatsApp community where we share information so you can join. I shared the link there as well and our Twitter and IG handles. Thank you once again everyone and have a good day, good evening. Bye. [laughter] >> Yeah. Thank you everyone. Stay nice.