About this talk
This talk explores the intersection of gaming and artificial intelligence, particularly through the lens of text-based games like Zork and role-playing games such as Dungeons and Dragons. The speaker, a passionate gamer and developer, introduces the concept of agents, which are AI programs that perform specific tasks based on user prompts. He shows how to build a multi-agent architecture using Strand Agents and TypeScript, highlighting the capabilities of integrating different tools like HTTP requests and bash scripts. The speaker demonstrates how users can create characters within a gaming environment, utilize an MCP server for centralized operations, and leverage an HTA protocol for multi-layer interactions. The session culminates in a live demonstration of creating a character and engaging in storytelling, showcasing both the technical implementation and the creative potential that gaming and AI offer.
Full transcript
All right, I will wait a few minutes until everybody settle in. Can anybody hear me? All good? Perfect. Perfect. Are you excited? How is it going so far? Good? Good. Perfect. Perfect. Have you played some games on the gaming area? Some of you. Nice. Nice. Whenever I see like arcade machines and so on, it goes like it just takes me back to uh my childhood, you know,
like when when I was younger, there was this arcade areas that you were buying tokens and playing like with our friends like Street Fighter for hours and so on. It was really cool. Of course, then we got spoiled with the consoles and so on, right? Like PlayStations, Xboxes, whatever it is. Like we had Sega and Ataris and so on. Crazy times. As you see, I'm a big
gamer. That's why today we have this session as well that you would be surprised how crazy of a gamer I am. Like I Whenever I have a chance, I play game like even uh even after I became became a father, I actually got myself a the like PlayStation Portable {slash} Nintendo Switch. So, whenever I have a chance that my son doesn't see, I can still play games.
And like I've been playing games since like I remember because like one of the first memories that I have is actually coming from this. I don't know if you have know this known this book series. Nice. Awesome. I wish I had the book with you so I can give that as well. So, for me the gaming is just all about story and the creativity of the people.
I love gaming so much that like whenever I have uh like I do something about it. We had but one of the best ones for me is text-based Text-based games are great because like it has been in our lives for a long time period, you know, like this was like choose your own adventure kind of book that you can have 22 different scenarios that you can live,
and it's just a book. And as we move forward, then we had more things in our lives as well. Of course, we had, for example, Zork, which was a more or less CLI game that you could play on your computer to follow on an adventure. And a bit more advanced version of of that came in later on as well. With all these names, some of you may
you might know you might know some of you don't maybe, but how many of you have heard of Dungeons and Dragons? Perfect. So, Dungeons and Dragons probably is one of the most famous role-playing games. They made the movies, they made everything about it. It's so crazy to me. And you might see you might see some dices here. Everything has a meaning. You are not using this classic
six-faced dices anymore. Every dice has a special way to be used. Like, if you're using four-faced ones, for example, it's for like tiny small actions or like potions on and more like more or less everything. The reason like the biggest reason though that I like all of these games including Dungeons and Dragons is I think it's best described by Sheldon Cooper from The Big Bang Theory. Like,
you don't need to have crazy hardware devices and everything. It runs on the world's most powerful graphic chip, it's the imagination. Everything that you see everything every story that you have is around actually whatever you can imagine out of it. right now we have an answer on why did I go through this way as well because I'm a gamer. But another reason was I want to show
you Zork. You can find this emulator online that it's basically you can see that like first you start the a project. You can say that, "Okay, open the mailbox." It says, "You are in the mailbox." So, you open the mailbox, and you see there's a leaflet. You need to open the leaflet or read the leaflet and see what is going on. And once you read the leaflet,
so first you need to take it, then you read, then it says welcome to Zork, blah blah blah. after a point it gets more complicated. So, when I say close mailbox, it says close it, then I say a very normal thing that I would understand leave front door. And it says you are not carrying the front door. I know I'm not carrying the front door, but from
the context it means leave from Then it's even crazy like when I say Uh even like leave the house, it says you are not carrying white house. Of course, I'm not carrying white house, but from the perspective of the story, it makes sense to it. Why? Because all the scenarios have been programmed into this. There isn't an actual big trained AI through large language models or something.
It requires you to have a direct match out of what you need out of That is why agents are coming to our lives. how many of you have not Okay, let me put it this way. How many of you use ChatGPT before? Awesome. So, if you use ChatGPT, it means you've used an agent before. An agent is a quite straightforward thing. Agent is a program that is
focused on an identity, and you give instructions to it to do something. In here, for example, we have an agent that is a dungeon master, and you give a prompt to it by asking it to do something, and afterwards, it goes to an intermediate thing, which is called large language model to process that information and bring that out, and later on call functions like either custom or
like functional functions out of the large language model to search the internet, whatever it is, to gather information, and later on return a response to you. In the most basic sense, you can think of agents like this. But if you take a step forward, you can see that it's actually like agent has a life cycle, a loop, you might say. So, when you a user ask an
agent to do something, first large language model check everything. Check whatever needs to be done and it says if there's a need to call any any tool or anything at all, then once it is called, it collects the information, then returns with the related information to you. Agent building is has been around all the time. You might have for example LangChain and Crew AI and so on
so forth. And another thing that we did is an it's an open source project called Strand Agents. And the reason that I chose Strand Agents is because of multiple reasons. One of them is it supports on TypeScript. I come from like I work for AWS, but I I'm not an AWS person. That's why they hired me actually. Like I come from a world of mobile app development
and web app development. So, like I come from languages like Kotlin, Java, TypeScript, whatever it is out there. And the support of TypeScript was the reason for me to go for this. I'm like, "Okay, I know what I want to do. I have the tool that I need. So, let's start building." So, what are we going to build today? At the end of the day, you will
see everything that you need to create a multi-agent architecture to create a character. And after the character is created, you select the gender, race, and the class of it and it it will generate a story for you to make actions and select like some basic actions or steer the story by yourself. And we will see a real demo of it. We are not just video bound. So,
let me just go back to my ID and we will continue from there. To build an like we will start by building a very basic agent. The first thing that we will do is by the way for the folks at the back, can you read this one? Okay, perfect. So, the first thing that we are going to do is to actually create an agent. And to create
an agent, the Strand Agents allows you to create it as if it's an object. So, you define an agent with a system prompt. A system prompt is an identity for your an agent that you define or that you create. You create the guardrails. You You tell what it should do or it should not do. And later on, you can either use any model, literally any model. You
can use OpenAI models, you can bring your own models, you can use the models from the cloud and everything that works with open or My brain was flying. >> [laughter] >> Uh like API key and so on. In here, I used the uh for for the sake of simplicity, I used the Haiku 4.5 from Amazon Bedrock. But one more time, you don't have to. You can just
use OpenAI models, whatever feels comfortable for you. And to invoke an agent, you call the invoke function with the prompt. This prompt is the user prompt. This is the ask that we have from the agent. So, I go back to my terminal. And I'm going to run this one. I will simple agent agent, yes. So, when I run this one, the first thing that it does is
it goes to large language model and brings back Oh, internet is not working. Nice. Let me quickly connect to the internet. If it doesn't, I have my uh 5G here. All good. It was working there. It didn't work here. All right. Let's see one more time. So, when I run this TypeScript file, when I run this TypeScript project, it will say I asked it to tell something
in the language or in the nation or even something that I say. It said I hope this is correct. Dutch. Like I I speak German, so it sounds similar, but I don't want to say anyway. So, I asked that it is a like dungeon master, so it creates a story. But of course, to move with the story, it asks me for more information. We are in a
good spot, but we are This was just one of operation. We will carry on with building more. to build more, what do we need? We need something called tools. Tools are making agents to work with some functions that you define. It can be custom or it can something that comes out of the box with the tools with the frameworks that you For example, if I go here,
one of the things that I can do is to do an HTTP call. When you use a coding assistant, for example, if it doesn't know the answer, there's a high chance that it will do a Google search. This is a tool that is working for you. And in here, I was I'm literally asking it to use the website to give me an information about who is the
creator or designer of the Dungeons and Dragons. So, I go back here and not this one. This is the way later stage. And what I will do is again NPM TSX tools agent. Ah. There we go. You can see that we created we created the agent again. One difference that we made was we actually gave in one array filled with tools. Now, we it only had one,
but it could have more. So, what it does is it tells me, "Okay, I did an HTTP request now. The tool is working." So, you would also see the same thing in your coding assistant that I said the I'm searching online. It doesn't do it behind the scenes. And it fetches the information, parses the data, and says, "These two people are the designers of Dungeons and Dragons,"
which is the correct information, of course, and it works out fine. Okay, we are in a good spot. what else you can do? Besides the HTTP request, you can do more things as well. One thing that you can do is running bash scripts. You can have a script running on certain occasions or when there is a match to certain scenarios as well. In here, for example, we
have a file editor in the bash and we say like you are cured gray hat. We explained it what it does and how it should be and say, "I want you to create a Fibonacci file with a JavaScript." And if I go back here, move it up like this and say that NPX TSX using tools, uh writing sorry, agent. Uh yes, writing agent. What it's going to
do now is it's it knows what it should do. So, it will information that it has about JavaScript and it will bring back stream that it has through a console log. It's quite straightforward, but it is normal. You can see that it created a file for me. If I pull this down, you can see that it has console log with everything here and if I go and
say that node Fibonacci.js, you can see that it is working somehow. Okay, we are in a good spot. We know we we have seen the file writing mechanism as well as the bash script to run everything. So, the next thing is we have been using the ones coming out of the platform itself. What if we want to create tools? What is the logic behind it? When you
want to create tools, you should always think about what is the function that I want to write like you would normally write in your programs in your applications. In here, for a dungeon master, for a game master, I need it to be able to roll a dice. So, for this one, I created a tool called roll dice here. The name is important, but the most important point
of a tool is a description. The large language model will take a look at what you want to build and have a match on the description to be able to decide on which tool to call. And here I said roll a dice with specified number of faces and when it happens it will do a very basic math random operation to be able to carry on. And the
dice master here again this is a system prompt giving an identity to the agent and the next thing is help me create a new D&D character roll the strength blah blah blah whatever it is done with the four 46 dice. With everything here you can see we created the tool and just passed it in like we were doing for the other tools as well. So I'm here
I say using tools agent with yes this one and if I pull this up you can see that okay the mystical dice chamber blah blah blah it just rolls the dice multiple times because every roll die every dice roll has a meaning and every with every dice roll it brings the information and actually makes a calculation for your character's stats or stat points and so on and
so forth. Okay so we know how to use tools and we how to build agent. The next thing that we want to learn is how we can centralize this information to connect more information to each other. And for this one we will be using something called MCP servers. If you have built a software with AI agents before MCP server is something that you probably use or heard
but for the folks who has not heard of it it's the protocol that is created created to give a service or a server that AI agents can understand large language models can understand. For example for our application it will look more or less like this. When the client asks for something first we will get the information about our tools, what is available. The next thing large language
model we decide on which available tool that is going to be used use that tool, and return the result. With the result being returned, then the large language model will shape up whatever information that is going to be pushed out. Why do we need an MCP server for this I don't want to copy-paste constantly the information. I want to have a centralized way of rolling the dice.
Maybe I will use this roll dice logic in a backgammon game, for example. This will allow me to push this information to the other platforms as well. So, let's see how we can create an MCP server. Um let me go here and close this one. first thing that we do is we are creating an MCP server from the model context protocol, the official libraries SDK. What we
have here is the name of the MCP server. Again, the name of the MCP server is very important as well, but it's also important to keep in mind which tools that we want it to work. So, in here in my server I registered a tool called roll dice like we did before. This time I made a difference. What I said was, I want to have the faces
and the count that are like how many times that I want to roll this one. And once I roll this one, you can see that it will do again a math operation completely through a math random. And that's it. It will be served through Express JS MCP server itself is created as MCP server, but to serve it you need to you need to have something to run
on. And in here we are using Express to define endpoints and so on. It's totally up to you. One important touch that I have here is I use the streamable HTTP. Why is it important? In the early days of ChatGPT and so on, you might remember you might be you were asking something and waiting for a whole lot of data to come back. Then later on we
had something called stream API. And with the stream API, now you have every answer one by one, word by word, or even like letter by letter delivered to us. So, this way we can first of all a stream of data showing that something is working and more importantly, you would be able to react on the operation as well. So, with everything in here, now we have Next
thing is the MCP client. The MCP client comes from the framework that you use. In this case, it is strands, but if you were using crew with you AI, if you were using LangChain, they have their own MCP clients as well. With the MCP client, the most important part is to give the information about where this server lands or lives. Once you have this, we get the
tools like I said before, push the tools inside the agent, and later on just run it as we have. The rest of it is just right now I don't want to have a single of operation. I want to have a recursive operation here. So, I go back, open a new terminal. The next thing that I'm going to do is first run And later on when my server
is running, you know what? Like let me run this one over here because we will come back to it anyway. TS6 MCP implementation. No. Dice roll service. This should be running over here. So, now the server is running, I go back and run the client here. Game master client. You can see that now it will ask me like what do you want to do? I can say
roll uh D4 for example, and we'll roll it, and it will give me a result of the action as well. It says like yeah, the fate has been spoken, blah blah blah, you have won. And if I do D20 for example, it's just rolls a like 20 face one and says it also dramatized as well because this is how you play Dungeons and Dragons, you know. right
now I have an MCP server and on top on top of it I have a communication Next thing that I need is I need a bigger architecture because I promised a multi-agent architecture. For this one, I want to talk to you a protocol called a HTA. How many of you have heard of HTA Nice, a few hands, good. So, HTA was actually HTA came after MCP protocol.
MCP, you can think of it as your libraries that you would be pulling into your applications. HTA on the other hand is like a way your applications can talk According to their definitions, they will be able to understand which agent to call or which agent to get that information about. It's an open protocol. It has been built by our great folks at Google and it basically allows
you to have a layer on top of MCP to be able to have more multi-layer architecture approach. How is How am I How am I going to do this thing? You will see that we have the player. Player is what basically [clears throat] you. And player is going to ask a question and it will go to an orchestrator agent. The orchestrator agent knows everything that it needs
to know. It knows the MCP server exists. It knows all the agents that is existing. It is the one that is going to decide which agent to call, which agents even to call, or ask them to communicate with each other to define an operation as well. Once everything is there, the next thing that it's going to do is to serve back all of this information to the
front end that we have seen in the beginning. How we are going to run this one? So, first of all, we have seen the MCP servers was server was running, right? So, next thing that you know what? Like top left. Let's do this top right. And the next thing that we will do is we will actually see the multiple agents that we are going to create in
the scores. So, for this implementation first of all, we have a rules agent. The is there to get the information about the rules of Dungeons and Dragons. You might be asking, "How is it going to get the answer rules?" The answer is not large language models because large language models have cut-off dates. And Dungeons and Dragons can change drastically. For example, we have one PDF file that
you can see that is created in 2018. It's open source, it's open for everyone to check out. You can check it out as well. That keeps the basic rules about it. I I don't want to take the risk of large hallucinating to do something. I actually want it to have the actual information. For this one, we are creating a knowledge base. The knowledge base is a vector
database that is that we are going to be feeding into the agent. And to create this, what I did was I took the PDF. And from the PDF, I also took a vector database that is working really good with TypeScript and use Hugging Face Transformers to get that and use the mini LLM on my machine to turn that into a vector database. To do that to do
that, I created the screen script that basically going through the data inside the PDF and turning it into knowledge base knowledge base chunks in my uh in my database. And once it is there, the rules agent can actually go through that information. It cannot do uh string search. Of course, we can't do a string search on vector database, but what it does is it turns everything that
uh that we have asked from it to Let me go back. The float 32 array checks a match. When there's a match, gets this information back as the rule that I'm looking for. And the what the agent does is just the querying the rules. This is the like full focus of this agent. The other agent is the character agent. The character agent's goal is only goal is
to get those rules and according to the rules and definitions about the characters and so on, create characters. And once the characters are created, the game master orchestrator orchestrator will decide which MCPT call, which agent to wake up and bring everything all together. So, with these every with these, we have the rule agent, we have the orchestrator agent, character agent. Now, we will run all of them.
So, I go back. And I will say left bottom quarter and right bottom quarter. Yes. So, first of all, I am running my rules agent. Not this way. HJ implementation, rules agent, rules agent. Yes, this is one thing. The next thing that I'm going to be running is HJ implementation, character agent. the last thing that I will be running is, of course, my uh orchestrator agent that
is going to be doing everything for me. So, right now you can see that Yeah. All good. All good. Uh I should be even done. So, right now I have my uh orchestrator agent running here. I have the other agents running as well. Everybody is aware of each other. We're in a good spot. What I'm going to do now is I can do two things. One thing
that I can do is to run this on CLI, but since I've shown you a website, I will run it on the front end and you will see that I hope everything will work fine. So, I open up my NPM run dev to it. I will go back. Copy this. Right click All right. So, you can see that I can choose any name. I will say Sally
the Waffle Master and gender male, race you can like half elf and class for example paladin. And when I kick this off, you can see that first of all, the character agent is starting. It will first roll the dice to get the stats that I need. And after it gets the information back, it will collect everything and says, "Okay, now I have every information. I already know
the knowledge base as well." So, here is the character that I'm creating. And when I Just a second. Yes. You can see that like it created this Waffle Master blah blah blah. I will just change the story a bit. I will say I am flying to Istanbul now. Where should I take my skills? Very random. But, you can steer the conversation. And according to the conversation is
like out of spo- sco- question. Let's see. I appreciate the question, but I'm a drill. It understood it as if I'm asking for some suggestions. I will say as paladin I want to move this adventure to Grand Bazaar in Istanbul. Let's see if it's going to work. Ah, the creative twist. See, like it understood that I it understood me wrong before and with everything in place now
it created the story and actually took me to whatever it is whatever we can do in Grand Bazaar in Istanbul. So, we have seen everything. I will wrap up very very briefly. So, we have seen these. So, first of all, if you want to check everything out, the code is available on GitHub. And like you can just play around with it. You can use your own model
as well like I said. The strands documentation documentation is over there. If you want to stay connected, please add me on LinkedIn. I'm more than happy to answer your questions. Before I wrap up, I want to ask you two things. One is I have the survey that I collect feedback from you to get better and everything. I really appreciate it if you can give any feedback. If
it's positive, I'm happy. If it's also room for improvement, I'm also happy. And the last thing is do you mind if I take a selfie with everyone cuz it would be cool. All right? If you don't mind it, find me and I will glue your face. All right, folks. One, two, three. Awesome. Thank you very much for being here. Thank you very much for listening to me
and have a nice day. Bye-bye.
