About this talk
This session focuses on security measures necessary for implementing authentication and authorization in AI agent applications. The speaker, Jeremy from Auth0, discusses how AI is transforming software development, particularly through natural language interactions, which introduce unique security challenges. He highlights the importance of addressing three essential questions: how to authorize user interactions with AI agents, how to control agent actions, and how to protect sensitive data. The talk also emphasizes the OWASP Top 10 vulnerabilities for large language models and underscores the significance of fine-grained authorization and safe token management. By leveraging OAuth extensions like Client Initiated Backchannel Authentication, developers can implement human-in-the-loop mechanisms to ensure security when AI agents perform sensitive actions.
Full transcript
And uh welcome to this session. My name is uh Jeremy and I'm a solution engineer at Auth0. And in this session I want to talk to you about security, obviously, and more specifically how to properly implement authentication and authorization for the AI agent that you may be building out of in the future or that you are building right now. AI is obviously a uh huge topic. Uh
it's changing the way we are building software. And my goal for this session is to highlight the uh security challenges around building agentic applications. And uh also more importantly how to solve them. So, how exactly are AI agents changing the way uh we work and build applications? First, human languages, like English, they are becoming programming languages. We can very simply instruct complex systems with natural conversations. Uh
this allows us to build agents that can eliminate repetitive tasks across multiple application even if you don't have any modern APIs uh in between. These agents, they can also run autonomously. In the background, they work for us while we are not even behind our laptop, while we are away, they just they just work for us. And they can coordinate step flows. They use memory, context to complete
uh the goals that we uh gave If you look at the uh the uh diagram here, you can see more more or less of a um typical agentic applications. You have a user prompting some uh uh a text. The agentic application is leveraging an LLM, and you have built as a developer or you're also leveraging some agent orchestration and some tool executions that will then communicate with
APIs from different SAS provider potentially some data databases or vector databases. And in the end when it gathers some some data to enrich the prompt to augment the prompt, um it may be that it returns an output or it executes an actions for you. And so the the the the agentic application will call these external APIs on our behalf, right? It will perform actions on our behalf
and this is where things get interesting from a security perspective. This architecture that we've just seen and we'll get back to it in in in a bits, it forces us as developers to think about three fundamental security questions. How do you ensure that only authorized users can interact with your AI agents? How do you control what action an agent can perform on a user's behalf? And how
do you protect sensitive data and credentials? If we answer these questions properly, then we make sure we build secure AI And we see these challenges and it's not only us at Auth0 that see that, we also see it in the community. In the developer community actually. And that forces us to rethink about what identity means. Obviously, we authenticate users, we authorize users in in our regular applications,
but now it's not just only people performing actions, it's actually again AI agents acting on our behalf and on the behalf of those people, of our employees, of consumers. And in some cases the agent is also acting by themselves. So we see a shift to from human-driven actions to agents-driven actions. And how do we control that? Because, yeah, we are less in control. We let them do
things for us. So, that's breaks the barrier of our traditional security models. And we from a user authentication, we cannot we cannot assume anymore that's just a means that we can execute all subsequent actions and that we stay secure. This is a slide that you probably see quite often. This is the OWASP top 10 for LLMs, for large language model. And yeah, I think it gives us
a great overview of what the threat landscape could be. You have things like prompt injection, where the attacker can hijack the LLM to force them to force to follow specific instructions instead of instead of the one that we that we gave. Sensitive information disclosure, that's when the model accidentally leaks private data. You get it too much, you deliver too much, you were not about allowed were not
supposed to see it. And there are other like other threats like excessive agency, it's when an LLM gets too much power, too much permissions, basically. And it unforeseen actions. It's it's a lot, but luckily most of them are it's the OWASP for LLMs and most of these concerns are for the ones building LLMs. Um if you build the agentic application, then you leverage the LLM and the
concerns are a little bit less. That said, doesn't mean that they are less important. But we can focus on the one that I highlighted here. These are really the vulnerabilities that come up in the AI applications that we build around the model. This means that they are within our control. Um as developers we have the power but also the we have the duty but also the responsibility
to uh look at this risk and mitigate them as much as as possible into our own code. So again, sensitive information disclosure, uh improper output handling when the application accept and LLM's responses without proper validation. Excessive agency we talked about the thing system prompt leakage, um vector and embedding weaknesses so that you bypass the standard access control to reveal sensitive data and unbounded uh consumption. That's when
you it's like DDoS attacks. You forces the LLM to use too much resources and that will cause system disruptions. Today we are going to focus on only these two again trying to narrow down so that we stay focused on on really two two uh on on on on only few things, excessive agency and sensitive information disclosure. Why these two? Um because they potentially represent a massive attack
surface with a potential for major damage for for the company or for for whatever services that you are providing and also for your users which is obviously the the most important consumers and employees. If an agent has too much power, it could uh see data, too much data, uh delete critical data, spend money without approval, it could leak sensitive information and then you have some serious issue
with uh compliance, privacy violation. So we need to focus on these and solve uh address uh address these these type of um risks. So, how do you secure how do you how we can secure this? It comes down to solving four uh engineering problems. First, authentication. We need to know who the user is. It should be certain. We need to authenticate them. Um secure token management. We
need to safe way to call the APIs that the that our agentic application will interact with. Because they will do that on the user's behalf, right? We need to control how the agent will interact with this API. Asynchronous authorization. Uh we probably need to ask for user permissions. Again, when we will instruct an agent to uh maybe book the best flight for us at the best price
or um perform any types of sensitive we are away from our computer. We are not um actively behind the prompts. Um the the agent is working for us and at a certain point the agent will need the user approval in order to continue. And the last one, uh fine-grained authorization. we need to ensure that the AI only sees and accesses what the user is allowed to access
and not more than that. And for that, we need a uh solid and granular authorization back to architecture. I I told you we'll go back to it. So, it's same, but then this time with these um four capabilities. User authentication happens at the at the front door. not not always. You could force the user to authenticate before accessing the agentic application, but it also can happen at
any time. At a certain point, we the application is more open and at a certain point, we need to know who the user is. We need to authenticate the user. So, user authentication, token management, which is the secure vault between um our uh execution, and APIs, that's or or any types of services, content that it will uh reach out to. Um, asking authorization, that's the we'll go
back to it also in more details, the human in the loop part, the agent is about to perform an action, is sensitive action, and we need user permission. And fine-grained authorization, uh which is um uh the the basically the gatekeeper uh for our data that is being returned to the LLM, making sure that we filter them properly, and then the data that is returning is only the
data that this user is supposed to view, to see, to access. It's going to make more sense in a bit. We are going to deep dive into into each of these. um, it all start with user authentication. Let me display a few more things. Yeah, that's perfect. So, we have a obviously a user that needs to have access to a resource. It's via an application, an agentic
application, but in short, that's what we need to do. We need to know who the user is in order to uh to authorize the user to see it. And for that, we need to authenticate the Um, with Auth0, uh user authentication is provided Um, you get a login screen. basically, uh you use Auth0 as the front door for authenticating your user. And since it's the front door,
that means that um this is also a place where you can add an extra layer of security. So, Auth0 will provide this login screen, but also things like bot detection, uh breach password uh prevention, this type of security uh measures. Users stay. Uh obviously, this application can be a mobile app, a single-page application, a regular web app. We have all the SDKs, all the uh APIs available
for you to quickly add authentication into your app. The user could be authenticating with different mechanism. If it's an employee using an AI agent in order to help customers, they might be authenticating with their enterprise IDP. If it's a tool directly for consumers, maybe they want to log in with social social provider. Um or you could be having the username and the password into Auth0 in a
custom database that you in a user store that you already have on your own premise. Or maybe you want to authenticate with passwordless and for that you leverage passkeys. All these functionalities are provided out of the box. The goal and also in terms of security, what I forgot to mention is MFA layer enrollments and prompt for multi-factor authentication. In some cases, maybe for a step-up step-up authentication
is also something that's that we can deliver in our actually what we call authentication pipeline. In this pipeline, it might be that we need to execute an extra logic. It can happen before the registration because obviously this actually is not only authentication but also has some sign-up possibility, reset passwords, and obviously authentication. So, pre-sign-up, post-sign-up, during the login, during password reset, and some more. You can add
these extra in order to, I don't know, if the user register, we also want to push this user to the CRM. We want to register that that user also elsewhere. If we need some extra data in our access and ID token that is supposed to be delivered to the application, and want to enrich that with data from the marketing tool to know what consent has been given
or we have a custom database with some more roles and permissions. We need to grab them, add them into the access token and in the end deliver this to the application. So this extra logic can be added into Auth0 during the authentication on runtime. If you are missing few informations and you need to progressively enroll the the user profile so in basically to add some extra extra
data about the user and you need to prompt the user for that. Auth0 forms allows you to create these interfaces and ask at the right time extra data about your customer. And finally, depending on the logic that you implemented during that pipeline, the modified access and ID token is delivered to the application. So our application knows who the user is, potentially what are the roles. We also
some marketplace integration to also help builds with other tools. The application knows who the user is, potentially what are what is the the role, the permissions for this user so that we can continue with with the journey. User authentication and now what happens post authentication. So let's say the the user logs in with with Google. You grab the off access token and the refresh token that you
just you just got and you hand them over to the agent's memory or you store them into the app local state. Very bad idea. Two main reasons for that, security and state management. First security The LLMs they are sensitive to prompt injection. You've seen that. And if you give agents a long lived access token and there's a malicious prompt that tricks the agent into dumping the context.
Then, with that access token, you can get a lot of access, a lot of permissions, maybe full uh including the user the user accounts. So, that's from a security perspective a um a bad idea. And the second one, um yeah, building uh uh uh this is it's actually hard to maintain. Um LLMs, they are supposed to be uh stateless. Uh access token, they are supposed to be
to to expire, to be short-lived and expire. And the LLM has no idea how to perform OAuth um uh refresh token flow, exchanging a refresh token for a new access token to keep that short-lived um And that means that if you start going down the path of implementing that yourself, then you need to basically uh implement OAuth, which might represent a big uh a big amount of
work, a lot of code, just to manage token states and keep them alive across the different providers. Cuz here we are talking about Google, but could also be the CRM, the Salesforce, any other APIs that you that that your application needs to interact with in order to perform actions or gather data. Solution to that is the token vault. So, again, instead of your application managing the OAuth
hand-handshake, storing refresh token in your database, we abstract all of that. We have token vaults, um all the sensitive credentials, like the refresh tokens, they are stored and encrypted securely in the identity provider, in Auth0, and it never touches the application or the agents. When So, so what happened is that uh when your agent execution needs uh to call an API, it simply makes a server-side request
to the to the vault, to Auth0 saying, "Hey, give me a uh connection token for the user's GitHub." For example, the vault checks if a valid token already exists. If it expired, then the vault will automatically uses the refresh that use the new refresh use the refresh token to get a new access token and returns it to the to your back end. So, that way your application
never handles that that logic. You don't need to handle that code, and that's helps you to just focus on the business purpose of that agentic application and not on this authentication and This is what it looks like in the code for developers here in in in the room. So, just to show you how the how it abstract complexity, you can just call this a get get access
token for for connection with the connection name. So, that means I need an access token for this Google connection. You get back a scoped short-lived access token. You pass it to your client, and you can then execute the tool. You can interact with the API. That's it. Less code that you need to write, less issue, less problem, and secure security without without bothering about about the complexity.
So, that was about the token vault. Now, our agent is able to securely call APIs. But, what happens is the if the agent needs to do something highly sensitive. For example, it wants to execute a stock trade. It wants to in the financial sector, maybe to transfer some money or simply email a contract to a customer. We don't want to let the LLM do that autonomously or
at least not every time. We need for that what we call human in the loop. Obviously, as a developer, it creates an UX in to provide that that approval UX and UI uh to your consumer and also from an architectural perspective, you need to think about, "Okay, how am I going to make it work?" So, if an agent is running in the background, running a task in
the background, and a user probably already closed the laptop, is doing something else, at least not behind the the the actual not behind the browser, how do you securely interrupt that background process that is running, reach out to the user on a trusted device, make sure that it's the right user, and ask for consent for a specific action. And then when this consent is given, properly communicate
back to the to the agentic application so that it can go on with the task. To solve this, we use a OAuth extension called CIBA, uh client-initiated backchannel authentication. And that's made exactly for that purpose. So, it's standard-based and it's made for that. So, think about the OAuth flow for for for the ones that are that are maybe not familiar with that. If you do authentication, you
rely on a browser, right? The user clicks on a button, login, gets redirected to a login page. Uh here you enter your credentials. You are hopefully uh uh successfully authenticated, and then you you go back to your to your application authenticated. This is a front channel flow. But the AI agent, it runs on a server. Doesn't have a browser that it can redirect to. So, uh CIBA
is the backchannel flow. The agent's back-end server talk talks directly to the authorization server and says, "Hey, I need this user to approve this action. I'm I'm now I see I need to have a to to to to break and ask for consent to that user." The authorization server then gets that information, reaches out to the user via notification, SMS, email, maybe at its best a push
notification on the user device. And so that the user can approve. Especially in the so so it's for sensitive data. So we quite often I discussed with financial institution on that on transferring money during this type of sensitive action. And if you combine that with another another standards called rich rich authorization request, then you don't you're not only approving a request, you can also send highly contextual
data. So potentially data about that transaction. Do you approve that agent to buy I don't know 10 shares of that specific company? And then the user sees exactly what it is about and can approve that action. Again, let's look at the codes. How do you implement that with Auth0? Here we define a a buy stock to stay on the on this the buying share example. We have
a best buy buy stock sorry tool for our LLM. You can see the the wrap the the the the the wrap the the behind the tool the core logic is in a in an Auth0 function called with asking user confirmation and that function does actually the trick. When the LLM decides to trigger this tool to to go ahead with the action, the execution the execution actually And
hits denied and obviously with that notification the UI is taken care of by all zero. If the user denies then we can send back a friendly message to the to the LLM saying, "Sorry that's that that was denied." And if it approves, then the code inside wrapper continues executes and the trade is made based on our example. So, again, way way simpler, it's clean, and it's a
secure way to handle step-up authentication for your agents. Okay, so that brings us to the last piece of of the puzzle, authorization. How do you make sure that the agent can't cannot overreach and return information outside of the user's This is especially critical with with with rag retrieval augmented generation type of of databases or pipelines. So, to to give the LLM context, rag then retrieves information from
your data source. But rag itself, it doesn't really enforce permissions. It's just what it's what it's being asked to return. So, without proper authorization, this system could retrieve sensitive data and maybe should not see that. So, you pass it directly to the LLM and then it's up to the LLM to display it. Yeah, so you need to you need more control on that. Usually you would say,
"Hey, I have a role-based access control in place, so I can solve it with that." But for very complex applications, it might not be enough Uh, simply because it's too coarse. and we are going to see an example for that in a So, let's take that example. Imagine a AI application for a hospital and the doctor asks, "Hey, summarize this patient's last 24 hours and tell me
how his vitals are reacted to the new medication." Okay, the right system will go, retrieve patient charts, patient information, nurses notes, lab results to augment the Now, how do we secure this? We could use our back. We could say, "Hey, we have roles like doctor. We have a role for uh nurses." But, what if the doctor is a cardiologist? Um they may be should they may be
should not see all the documents that are being returned because this doctor is not responsible for that patient. So, you can see that the simple uh role of a doctor is too coarse. It doesn't understand the specific relationship between in that example, a doctor and a patient. So, this is why ABAC is not enough and access is is isn't just about a user job title. Sometimes it's
more granular and more about the relationship between that that piece of data to be a little bit more generic and step away from that doctor-patient relationship. Can obviously not apply only to Agentic, but any application, any complex authorization model uh is at a certain point necessary in order to define that complexity. Talking about the authorization model, this is where I can start talking about fine-grained authorization or
FGA. Um it's built on the same model. I'm not sure if you know Google Zanzibar for uh it's what Google Docs is using for sharing documents, calendar, etc. And it is made to design to to design and and answer this complex relationship-based issues. So, you have a authorization model that you can define. Um you will end up with a relationship graph to have a visualization of in
that example again this doctor and patient relationship. And in the end these relationships are defined as tuples in the in FGA. And FGA also delivers a lot of APIs in allow other maybe more business people to define that model and and and and update it if needed. just to finish quickly on the on the >> [applause and cheering] >> So, FGA will when the when the when
the user make makes a prompt and show me the last appointment record or show me that that patient's medication reports, we will retrieve this data and also FGA will basically filter these documents and answer back with only the filter set of data and only the documents that the user is allowed to see. So, everything is properly scoped and the LLM is not making too much usage of
the documents that the user is maybe not allowed to So, to wrap up, if whether you are building agentic applications or AI applications, you can start with Azure for free. Here you can create a free trial. Also, if you would want to uh a little bit of hands-on, we are organizing a dev camp on the 4th of June. We'll give you all the tools to build to
to spin up a agentic application and to build all the use cases all the four capabilities that we've seen in that session. Thank you and enjoy the rest of your day. >> [applause]
