Deploying AI Agents with Strands Agents SDK and Amazon Bedrock AgentCore - Runtime Vishal Alhat

This talk explores the process of developing AI agents that can be successfully deployed to production. The speaker, a developer advocate, emphasizes the importance of transitioning from a local environment where applications may work seamlessly to a production environment, where challenges such as performance, scalability, security, and governance must be addressed. He introduces two key services, Amazon Bedrock and Agent Cortex, which facilitate the building, deploying, and operational management of these agents. Through practical demonstrations, the speaker showcases how developers can utilize a model-driven approach using the open-source stance agent SDK to streamline the development process and manage various operational aspects like security and evaluation. The session concludes with best practices for ensuring reliable agent performance in production environments.

When talking about developer advocate, I'm someone who is like you guys. How many of the people are developers in this room? Almost everyone, right? Many of the people are developers, many of them are business leaders, decision makers in the organization who want to learn about technologies, AI, and well, cloud operates with these AI agents, okay? So, today I'm going to help you with understanding how we can

build our own agents and get that agent to the production. Building is simple, you know, we all are using AI, you are building the applications, but nowadays, uh when it comes to getting those things into the production, it is very different. We need to take care of many of the things, and we are going to see what are the things that are important today, okay? So, as

a developer advocate, I help people to build applications. I'm more focused toward development strategies instead of just talking about the products or services. I I usually talk about the pain points, or maybe the problems that developers are focusing and how we can solve that, okay? So, today we are going to see a few of the things how we can create application prototype, agent prototype, and how we

can take it to production, okay? So, I'm going to talk about two of the services that are uh Amazon Bedrock and Agent Cortex. I'm going to show you this. So, this is going to be more of a hands-on session or a demo where I'm going to show you show you how we can build those things, and then how can we can take it to production, okay? So,

let's start with that today. Okay, let's start with a story, quick story. We have all lived this story, Uh we were building a AI agent. We just tried to start with the AI agent, building agent to agent on weekend. We called some LLM and we just use couple of tools to build that. We beautifully created that. It is working perfectly on the terminal or on your UI.

Okay? And it's already you are tested that with various use cases or test cases and it is working over there. And then you try to deploy out on production for some on cloud or some data center and anywhere that it's your choice. So, at the time you see that it is not working. What could be the reason? How many of the people that are familiar with DevOps

over here? DevOps strategies? Many other people? Okay, that's good. So, in DevOps you might have heard this term works on my computer. Right? Works on my laptop, works on my computer, it's not my problem if if that doesn't work on production. So, in this kind of scenarios always code works on your local because you have built it, you have tested it, but when it doesn't work on

production there are some things that you need to take care of. in the agent that you have built on the weekends we have built it for local ready, you have not built a product for production ready. So, we are going to see what are in order to get that production ready. Okay, so in order to get the application or your agent from production prototype it looks very

simple. There might be some DevOps pipeline, there some might be some CICD pipeline that will help you to create your CICD and then you can deploy that easily on production. It's not that easy, right? So, you have to go through many of the steps over there and uh when we start with prototype, if we are like we do a POC. Obviously for any of the tools we

do a POC. That POC works very perfect. Everything is very very perfect, there are no pain and then it's time to take it on the production and then you see those challenges, right? So, what are the challenges? First one is performance. Because you are working on local system, you are the only user or maybe your team four or five people you will be working that. When you

try to take it on production and when 100 or millions of users hits it, it breaks down. Okay? So, performance is one of the things that we have to take care. Scalability, maybe it is working for one reason one region or maybe for particular number of users, but it is not working for huge user set or maybe for multiple regions. That's where the scalability come to picture

that we have to think about. Security. maybe I have specified this as a challenge later on after the development, but when I talk about the security, security is always a day zero job. Okay? Always a day zero job. So, I can see many of the people are developers here and I always talk about the security whenever we implement the first line of code, we need to make

sure that we are aware about the security. Few of the very basic things, never store your credentials in your code in your repository. Always use credentials managers or any secret management platform in order to store things. It might be your database credentials, it might be your API access credentials, anything. You can store in SSM any any of the SSM services and use it, refer it from there.

Okay? So, security is the one of the things that we have to think when we are on path to production. Okay? >> [snorts] >> And then governance. Governance is more authentication and authorization for your application so that only the genuine users can go into your system. No hacker can go into your system. No unauthorized person can go into your system. And there are all particular guidelines guidelines

guardrails set for your application. Okay? And when it goes and it you have done all those things and it's production ready, uh this this cousin really it's it's really deep. It takes time. It's not that easy. When you do those things and you have completed these all requirements, your application brings uh creates more business value for your organization, Okay, let's talk about the evolution. I'll give you

some context where we are right now in this evolution of AI agents. It started with in 2023 uh where there were chain basically evolutions where there are hardcore chains of the for each of the agents, okay? And then there were fixed execution part. It was not dynamic. It was not A-to-A communication at that time. And then there were complex DAGs for each of the Uh there were

some challenges uh when it was uh working for multi-agent systems. You had to call APIs and do those things. Later on in 2024 we saw orchestration orchestration of agents where we saw the multi-agent communications uh happening smoothly. And uh uh like with the simple use cases we could see that there is rise of heavy uh orchestration tools that are coming into picture in order to do those

in model-driven way, okay? Uh and if you see mean modern LLMs like Claude, Noah, or any of the LLMs that we are seeing that are capable for reasoning, uh we have seen that those tools are calling to the models and external models and doing the orchestration on their own. You don't have to do those efforts to do those things. So, uh we are we have seen in

2025 in current model-driven era, uh LLM as orchestrator we have seen uh many code. You don't have to write long code. And I'm going to show uh show a demo how quickly you can create agent uh in a simple way. Uh we can call any native tools and it's so simple to understand. Any non-tech person can even understand what is written in that particular code. Let's move

next. So, uh AWS has introduced a open-source service called stance agent SDK. It is a kind of open-source framework that is released under Apache 2.0 and with more than 5,000 stars on GitHub. It supports both Python and entire type scripts. So, it have you have freedom to use your choose your own technology. And then core philosophy of behind this is it is model driven instead of writing

complex orchestration code, you can do it in model driven way. And with this with just 10 lines of code, you can build your own agent. You don't have to write complex API calls, complex code to do those things. You can even integrate that with MCP servers or you can use built-in code interpreters to do the operations with your agents. And it works with many of the models

like you can connect with Amazon Bedrock or maybe direct anthropic API calls. You can use open AI open AI to connect with your local server as well with Ollama. If you have Ollama using Ollama, you can connect with the local server as well. Okay, so here is the building an agent with strands. This is how it looks when you are trying to build your first agent with

your Bedrock and strands. If you see little it is it is a complete code. If you if you can see it on the screen and you can see very few lines of the code. It is a complete code and it's a working agent over here. So, here we are using Claude Sonnet via Amazon Bedrock. And the tool that agent can agent can handle is with the reason

it has reasoning loops internally where it decides where when to call the tools and how it can interpret the results over there. Okay. So, if you can see over here, no chance. There are no graphs and no routing logic. You can do those simply with this model as the interpreter and model will route those queries as per your requirements and as per your prompt that you are

passing on. Okay. so we saw agent. I know everyone can build their own agents and they can build their own agent that will work perfectly on a on local. Now it comes to the hard part when we have to build that agent for the production. So whatever we have done, we have created a code that is just a 10% when you talk about the agents, right? Because

there are lots of the things that we saw about the agent. So how do you package that particular agent? That is one of the things. What are the dependencies that are required for runtime environment for your application? We have to consider Then how do you deploy that particular agent? When you are ready, you don't have any devops strategies that you had planned earlier. You you need to

plan those things, okay? And do you need do you really need Kubernetes or you can do that with serverless platforms or on standalone virtual machines? You have to think about that. Maybe you can use uh platforms like EKS or maybe if you want to do that for on serverless platform, you can use lambda in order to orchestrate that thing, okay? After that, how you can scale this?

What happens when maybe 100 or 1,000 users hits simultaneously? On your local you just tested with group of users. When what happens if the there are hundreds or thousands of users are hitting simultaneously? In that case, scaling comes into the picture, right? in terms of security we saw authentication or we need to consider authentication strategies for user, how we can isolate the session for each user, what

tokens can we use, and how we can encrypt the data as well. So all those things when we are creating a production ready application, we have to think. Apart from that, in terms of SRE, observability is one of the tools where you need to know that how your agents actually perform in 2:00 a.m. in the morning when you're sleeping. Uh so if there is something that is

going wrong, you need to know. So there are logs or traces that you can go and uh debug along with that logs, okay? And then evaluations, how do you know that your agent is actually performing as per your expectations? So, in that picture, in that particular case, AWS provides with you with agent core and science evaluations that you can go take care of, okay? And then this

is like once all if you stuck in any of those particular things, you have to go back, debug the things, and then check where it is actually failing, and then try to find out. It might fail in deployment, it might fail during the scaling, or it might fail during the evaluations as well. Because evaluations that you do manually on your local, that might work perfectly because it

is a very limited environment with limited user sets, okay? So, when you do it's it's if you have to do this checklist manually, if you're doing this locally and deploying that manually, it takes long efforts, right? For main painting those checklists and doing those things. What if you have a ready-made platform to do those things that can handle all of this all of those things on your

behalf? So, I'm introducing you to the Amazon Bedrock agent core that is in order to address all of those challenges, uh this is going to help you. This is a set of modular capabilities to build, deploy, and operate production grade uh agents that can be scaled using any framework any framework in the market. I'm going to show you demo for science framework today, but you can use

any of the frameworks in the market and build your own models and own applications. Okay? So, these agent core capabilities are optimized to work together or maybe individually as well as per your requirements, and that they give you to a flexibility to integrate that with any of the open source frameworks that you are using for your agents. >> So with this agent core framework, it is going

to help developers to optimize the time and like time to value for particular product so that you can focus more on development things apart from looking for the scaling, security, observability, and evaluation and all those things, okay? So, apart from that, those all of the agent uh agent core services are serverless. So, you have to you can pay only for the things that you are using. You

don't have to pay for particular services that are not in your uh architecture, okay? So, each service is modular, so you can plug in and simply use that. You can use that independently, or you can create your own infrastructure on AWS and use that along with the rest of the even if you're using OpenAI or LangGraph, we can integrate that uh with your agents uh from agent

core. And you can upgrade this uh now those agents with your own controls, own access management things. So, if you're using any third-party access management frameworks or tools, you can integrate that with that uh agent core. Uh so, overall with agent core, you can accelerate your production deployment for your agent that you're building with scale, reliability, security, and uh observability. Uh and that can be a be

a real-world scenario for your deployments for your agent, okay? So, let's uh go through quickly through the uh capabilities that we have. So, if you see this is core of agent core, this runtime that you are seeing in the diagram, it uses microVMs. Uh think of a fire- Firecracker if you have heard about Firecracker. It is the same technology that powers AWS Lambda serverless platform. And each

agent session that you see over here, each agent session uh gets its own microVM, a small virtual machine, which has complete isolation from rest of the VMs, and each agent session uh gets uh its own like if you see, there are multiple AB if you consider session or agent to ABC, the it will have separate isolation for the things. And this this runtime uh supports up to

8 hours session. So, this is pretty critical because when you are talking about multi-agent systems and you are connecting with the multiple agents, uh it is really required over here. like consider you're generating a code or you're doing a research with your agent and that needs to crawl multiple files. There are maybe a payload that you're sending maybe 100 MB of payload that you're sending for your

agent. In that case, it will support you with the multiple passes of the document. Because you can you can't pass 100 MB file to your agent. So, in that case, you can send it to the multiple passes, keep the session consistent, and do those things, okay? And apart from that, it provides you with the auto-scaling, load balancer, and all those things. You don't have to think about

that as well. You just deploy your code and agent code will agent code will handle all the things that are needed for you, okay? So, here there are two deployment options. You can zip your code and upload that directly or you can use your container image for more control on those things when you're deploying your code. So, this is a simpler and faster way to create your

agent using agent code. So, this is kind of agent code starter toolkit. If you can see on screen, this is demo that I have recorded. So, with simple agent code create, it is going to create a boiler plate or scaffold for ready to run agent project with agent code. It will have all the defaults that are needed for a simple agent. And it is going to ask

you the options uh as per your requirements. So, it just uh agent core create, it will create all the uh basic scaffold. It will support multiple agent frameworks, so you can connect with any of these uh agents. It has multi multiple ISC options. You can use cloud formation, you can use Terraform to create that. It has its own memory, if you want to start with local development,

you can type in command agent core dev. That will create your own development environment, and start with the local development loop, and then all the hot related options that you have in your development, okay? And when it's ready, uh when we have tested all those things, you can just hit a command agent core deploy, and it will deploy that agent to the agent core, and you can

connect it just in less than 5 minutes, you will have your agent running live on the systems. And every every of the step that you are following has the default set, it has the scaling mechanism, it has a security in built. So, you just have to run that, and if you see on the demo, it is going to create that, it has created the code that is

required for your agent, and then you can update that code as per Okay, then let's talk about the deployment journey we talked about. Uh is it clearly visible at the end? I'll just explain those things over here. Uh So, in the deployment journey, there are clear five steps that you are going to see. You are going to develop the things locally, then um as we saw in

the earlier demo, uh you can build your own agent, test it on your uh on development, or on your local machine. Then in step two, if you see you can package it up using zip code or maybe with all the dependencies or we can create our container image as well. Okay? Once it is done, if you need more control over your runtime environment, uh what you can

do, uh you can like with the container once you're done this, third step is going to the you are going to deploy that using agent core CLI that we saw how we can do that in the using the command. So, uh it's just a simple command agent for runtime deploy and uh it will deploy using CLI. It will create endpoint. It will provision provision all the infrastructure

that is needed for your agent and it will set up the the networking as well. So, you don't have to think about those complex things. You can just focus on your development tasks. Once this is done, step four is uh invoking your agent where uh through API endpoints you'll get the API endpoint. So, uh you can go you can get both synchronous and in synchronous invocations uh

with the API endpoints. uh um and after that in the step five you can monitor the with the built-in observability tools that AWS has. Uh the key thing here is uh what you do not see in the entire picture or in rest of the uh frameworks is there is no direct Kubernetes configu- configuration So, there are no Docker Compose files, no load balancer setup over here, no

auto scaling policies as well for scaling. And uh agent core is going to abstract all of those things. It will take care of those things. So, if you see uh many of the developers feel this as a pain point when they have to configure the Docker files or maybe Kubernetes clusters or security policies for their particular applications. So, all those things are abstracted away from you. You

can just focus on your agent logic that you are building. You don't have to think about rest of the things and it handles all the operations on its own as a platform. Okay, next one is security. Again, non-negotiable in the production when we are uh deploying those things in production and we are talking always about the production. The agents are really critical because agents can take actions,

right? We all know agents can take actions, they can do those things, they can call external APIs, execute code, access any of the data that you have. So, in that particular case, security really becomes very important. So, with the with the agent core implementation becomes very simple because you have five security layers Layer one is networking and API security through the gateway, okay? Where it comes where

it helps you with the rate limiting, DDoS protection, so there is there are no denial of service attacks on your agents. Uh it helps you with the API authentication. Layer two is I am where it integrates fully with I am and OAuth and OIDC support, so you can use your own authentication systems or inbuilt AWS authentication systems for for fine-grained authentication for your application, okay? Uh layer

three is compute isolation where we saw there are micro VMs that create each session. So, every agent is isolated or every service in the agent is isolated. In layer four is data protection where data is encrypted at rest as well as during the transit. So, you don't have to even think about the encryption logic and safety of your data when it is in the transit. Uh VPC

is going to support with private networking with strict strict tenant tenant isolation, so every tenant will be on a different uh will have the different isolation. And at the last, layer five, it is audit and compliance for each of the request that get data get getting logged. So, each of we can go back, see if there are any security issues, if there is if there if there

are any scaling issues, or if there are compliance issues, and fix that. So, this kind of security posture that takes months to build with your own agents or if you're doing those things manually, it will take months. With Agent Core, you can do it very simply uh with uh uh the built-in tools that Agent Core uh has, okay? Next one is uh really important because we have

uh um learned this very hard way productions uh production deployments with many of the customers and clients, okay? So, few practical lessons uh from running those agents actually in the These are the mechanisms actually. These are not the advices that you can follow, but these are the standard mechanisms that you can do. First one is uh invest in evals very early, so that when you are uh

before you're deploying or uh defining what success looks like for your agent, how your agent will be successful, set up automated evaluations. It should be not not be a checklist or manual manual evaluations. Because you are in a general world, you're doing the change changes frequently for your agents, set up those things in automated way. And try to measure accuracy, latency, speed, cost, security, and reliability Second

one uh uh traject- trajectory evaluation uh trajectory visualization. Uh when an agent gives you wrong answer, what what you need to do or what you need to actually see what it was thinking at each step. You have to evaluate that and correct those things if it goes wrong, right? Uh so that like it is kind of visualizing visualize what your agent is trying to do and trace

that thing back. Like you can uh see which tool it was calling. Uh consider scenario where you are sending the email from your agent, and there was particular scenario you were calling a service called Twilio or any external uh service that sends the email. Something went walk uh wrong, and uh it added some encoding over there in your email, or maybe the path for image was incorrect.

In that case, what you can do is you can trace back, you can see and visualize what agent tried to do. Revert back, do the changes, and update your code. Okay? Third one is feedback loop loop where you can deploy the small changes. I always suggest developers to small start small and scale up as you go. So, deploy small changes to your system, measure the impact immediately

with the evaluations, and use the features those particular feature flags to control the rollout so that you can do those deployments in a reliable way in a faster iterations. So, there is a key when you develop the agents. The faster the iterations, faster the agent code improvements are. The fourth one is intentional changes only. Where this is This is again huge because prompt drift is real problem.

Everyone has seen this how your prompts are drifting and how your accuracy drops. How your agent accuracy drops when the prompts go wrong. Okay? the fifth one is production sampling where automated evals cannot catch everything. You can do a random sample production interactions and have humans because when you are you are developing the agents, there should always be humans in the loop, right? Whatever you are doing,

there should be humans in the loop. It should not be entirely automated way. So, do human reviews and you'll find that failure modes are not just automation test that can catch everything. You need you know, always need human in the loop in order to test those things. Okay. So, let me leave with the three key takeaways that we have seen today. First thing is model driven agents

are really future for us. We are past past the era of the the complex chain based systems were existing and we are orchestrating those orchestrating those things manually, we are applying those security policies, we are apply applying those scaling mechanisms manually. You don't have to do that now. You can just focus on your agent logics, okay? Uh the stand stance SDK philosophy that we have seen are

is letting us to orchestrate our own models in easy way using bedrock agent core and we can really build your own agent start starting with 10 lines of code and then you can add start small, add your logic to the agent and build your own agents. Uh once this is done, agent core runtime is going to provide you with the so that you don't have to build

the things from scratch. You can start with small and go ahead, add your logic, iterate those things, verify the things, evaluate, check logs, do the observability, check the logs, trace those things, and come back with your solutions, fix those things, okay? everywhere everywhere everywhere when you do it, you don't have to wait for the problems. You can see those problems in runtime when you are testing those

things, when you are evaluating those things, and before even those the deploy actual deployment happens. And then you can build your own trajectory visualizations from the same start. You can do those things on locally and when you are going going that on production, you are going to do those things, okay? So, you can create fast feedback loops that we have seen and make them intentional when you

are doing those changes. So, those feedback loops, whenever you are talking on any of the agents, even if even if you are building a simple chatbot, very simple chatbot that is going to respond to your queries, you need to have this feedback mechanism so that it can learn from your responses, it can learn those things from the inputs that you are giving. And as a human when

you are evaluating that kind of system, we are going to see if that feedback is really helpful to your agent, if it is really uh help helpful for the inputs that you are giving because your agent might hallucinate at any time as per data set data set that you are giving and every production system might have different ways of interpreting that data. It mostly depend on the

data that you are giving, the models that you are using, and what are the inputs and what are the outputs. Consider a scenario of uh uh maybe a chatbot uh let's consider a legal chatbot that accepts some inputs about your legal cases. What you'll give? I want I have got this challenge from uh this particular system and I how I can get help from government about this

or how my lawyer can help on these things. It is going to go to the LLM, check what is the historical data about that particular case or that particular number of uh like if there any existing cases, it's going to see that and provide you the response. Will that be always accurate? No, right? Because there might be different scenario scenarios for each of that similar case. Um

if you consider traffic challenge, there might be some hit and run, there might be some signal jumps. So, in that case, that agent cannot be always correct. So, in that case, you have to manually evaluate those things with the random ways and try those things. So, with the agent core and stance, we have seen how it it is going to help with evaluations. Agent core has its

own capability to do evaluations and uh uh these are the best practices uh that will help you to save the teams who are trying to successfully run those agents in production. And from the people who are struggling to do the agents with the reliability. So, consider two different organizations that are trying to do do those things manually, all those uh scalability, security, and all those things manually,

they're going to take months to build particular agent. But if the team that is using uh uh agent core and bedrock, they'll build it in like a week and with entire evaluations and complete scalability, reliability, as well as observability for your applications. So, that's all for today. these are a few of the resources you can get to started with agents strands agents SDK and bedrock agent Uh

these are the GitHub These are the way you can install it There are documentations for the bedrock agent core. You can start with the CLI. You can use those protocols and these are frameworks with your application. It is open source, so you can use it. You can contribute back to it as well. And for If you're using Python, you can use this GitHub repository, clone it, and

for your applications or for your Thank you. Thanks for your time. If you have any questions, you can ask or I'll be there at AWS booth. If you have any questions, feel free to connect. You can connect me on on LinkedIn. I'm available on LinkedIn or Twitter. You can search for my my name. I'm Vishal Alate. You can search and connect with me. Thank you. Thanks for

your time, and have a nice day. >> [music]