How to Prevent AI Agents from Accessing Unauthorized Data - Sohan Maheshwar

In this talk, Sohan Maheshwar discusses the critical issue of AI agent security, particularly focusing on preventing unauthorized access to sensitive data. He highlights the growing prevalence of AI in various applications and the accompanying risks, such as data leaks due to insufficient authorization mechanisms. The speaker explains traditional access control methods, including role-based access control (RBAC) and attribute-based access control (ABAC), and emphasizes their limitations in complex systems with AI agents. He introduces relationship-based access control (ReBAC), popularized by Google's Zanzibar system, as a more effective solution for managing permissions. Finally, Maheshwar presents SpiceDB, an open-source implementation of Zanzibar, demonstrating its capabilities in managing fine-grained permissions for AI agents to ensure secure access to data.

Hey there. Welcome to my talk on preventing AI agents from accessing unauthorized data. My name is Sohan Maheshwar. I'm based in Amsterdam in the Netherlands. I've been working in the industry for more than 16 years now and I've spent most of that time in the cloud and I even spent time working on the Alexa team back in like almost a decade ago. So, it's fair to say

that I have experience in the world of AI and cloud and security. And today I'm talking to you about something that I think is very important, which is about AI agent security. Now, if you've been seeing the news of late, AI is everywhere. AI agents are everywhere. Uh but also things like data breaches related to AI agents, well, is everywhere. I just thought I'd pull up something

from the news, right? Um This was about a rogue AI that led to a security incident at Meta. And there are so many stories, you know, very similarly where an AI agent accidentally leaks data. And the key in all of these articles, you will find one word, which is something you will see in this article as well, which is sensitive data that they were not authorized to

view. Because this is such a problem, this fact about authorization for AI agents. And of course, I work in the space, but it's not just me. It's the data around it as well. Um if you've ever checked in code at any point of time, you might have heard of something called OWASP or Open Worldwide App Security Project. It's a foundation that looks at things like risks to

web apps. And every 4 years or so, they publish a list of the top 10 risks. The last two lists, including the one that came out just a few months ago, is topped by broken access control. And here's a scary stat from OWASP itself, that 100% of apps tested had broken access control issues. Now, this data is from, I would assume, the last seven or eight years.

So, you can only imagine how much AI and agents and rag and LLM and MCP compounds this particular problem. So, if you weren't alarmed earlier, now you should be, which is why I think this talk might be useful to you and might save you from the next big data breach. And I think AI makes this a lot more harder because of what I like to call ambient

context. In the past, when we made authorization decisions, it was fairly simple. It was, oh, is does this person belong to this particular role, or do they have, like, say, something like an access token? But with AI agents, there's this thing called ambient context, which I like to think of it as this evolving graph of relationships, states, and entities, you know, between users and agents and resources.

And it's hard to contain them in something like a jot token, or in, like, a request payload. Like, you're making a call, how do you, you know, have all of these relationships into this one payload? You don't want like a 50 MB payload, right? And just think of the scale of this. Your company has N number of users, and you probably have M number of AI agents

running, and you have, like, say, O number of actions. So, you can't really create like a role for N into M into O, you know? So, past authorization paradigms, such as role-based access control, really don't cut it in this day and age because our systems are so complex and we're using things like AI So, what does actually cut it? Well, for that I might have to give

you a bit of a history lesson in the world of authorization. And very quickly, when we say auth, people conflate authentication and authorization, and they're two very different things. Authentication is identity, who you are. If you've ever used things like LDAP or Okta, that is authentication. Authorization is what comes after that, which is permissions. We know who you are, but what are you allowed to access? And

in the past, like I mentioned, we have used things like access control lists, which are giant lists with all the people who can edit document one, and another list of all the people who can read document one. Not super scalable, of course, but the most popular one, which I'm sure you're familiar with, which I even mentioned just now, is role-based access control, or RBAC. You might be

tagged to a role like software engineer in your company, and all software engineers can access the code base. But now, with our complicated systems, where you have hundreds, or if not thousands, of code repositories, you don't want to open that up completely, right? How do you actually, you know, make sure that a software engineer can access only the repos that they are supposed to access? So, role-based

access control doesn't really scale to that unless you want, you know, millions of roles. There's something also called access-based, sorry, attribute-based access control, or ABAC, which whittles everything down into something called attributes. So, your name, your location, your email ID, your IP address, all of these are attributes, and at the time of decision, all these attributes are checked, and a decision of yes or no is made

based on that. Also, can be hard to scale and can get very complicated in complex systems. So, none of these really work well for an AI agent use case because of the reasons that I just mentioned. At this moment in my slightly biased opinion, but also what you're seeing in the industry right now, the thing that seems to be working is something called ReBAC or relationship-based access

And this is fairly new and it was popularized in a Google white paper from about 6 years ago. And this white paper is called Zanzibar. Now, I highly recommend you actually read this white paper. It's about 14 pages long and it describes Google's globally distributed authorization system. So, have you ever used a Google product at any point of time? I would assume yes. And if you have,

the system that works behind the scenes to authorize you to access a document or a YouTube video, photos, maps, etc. is called Zanzibar. And this white paper describes exactly how the system works and the fundamental approach is ReBAC or relationship-based access control. What is that? Well, instead of shoehorning everyone into a role or breaking it down into an attribute, everything is described in terms of relationship relationships

between For instance, a person can view a folder, so there's a relationship between the person and the folder. That folder has documents, so there's a relationship between that folder and documents. And hence, by transitive property, there is a document. Cool. So, what are the things that can have relationships with one another? Well, in a Google Zanzibar system, one has to be a user and if you have

a Google account, you have something called a Gaia ID. So, that has to be one thing. And the other can be anything. It could be a folder, a doc, a photo, a sheet, map, link, whatever. Right? As long as it's user and a thing in Google system. And what kind of relationships can be between these users and these objects? Well, again, anything that you define. So, for

instance, member of a group or editor of a document or uploader of a video. Cool. So, we have these users, we have these objects, we have the relationships in between them. What do we do with this data? How do you tell the system that these things exist? The Zanzibar paper describes something called a relation tuple. Now, this is the format by which you tell a system that

a relationship exists between two things. So, start on the right-hand side of the hashtag and go user number three is the owner of document 1 2 3. And this is a very flexible format objects can also relate to other objects relations. So, for instance, group security is a member of group engineering. And think of the time you've had a reorg at work and, you know, say group

security suddenly reports to group devops. Imagine all the things that happen behind the scenes for that to happen, but in a reback system like Zanzibar, it's just about creating a new relationship. All right. So, we have these users and objects and relations and tuples. What are we really doing with all of this and why is this important? The idea behind reback is to break down or represent

these relationships in the form of a graph and you're breaking it down into a graph traversal problem. So, I'm going to give a simple example of something like Google Docs where you have some document, you have people who can read the document, Fred and Kim, and people who can write to the document, in this case, Jill. The key thing here is this little dotted arrow, which indicates

that all people who can write to this document can also read the document. And this is very flexible because say your requirements change and you want to add something like an admin to Google Docs style application, well, it's another node in the graph. That's it. So, you have an organization called Acme Inc. and you have an admin called Niv. And again, see the little dotted line, which

indicates that anyone who's an admin can write to the document. >> [snorts] >> And we already know that all writers can read this document. So, say you click on a Google Docs link and sometimes you see this access denied, what is actually happening for you to see that? Well, it's a simple graph traversal problem. If you can go from the permission, the check is to see if

Jill can read some document. Right? So, the idea is if you can go from some document to Jill via the reader permission in a unidirectional way, that means Jill has permissions. So, in this case, let's take a look if that is possible. And well, yes, that is because you can go from some document to reader to writer to Jill, unidirectional. Same way, let's check if Fred has

write permissions to some document. You tell me. Do you think Fred has write permissions? Answer is no, because you cannot go unidirectional from some document to writer to Fred. And hence, if Fred tries to edit the document, he gets a thumbs down. The question I get asked is, "Okay, all this is good. Um what about performance? How does this scale to the scale of Google, you know,

billions of documents in pretty much every country in the world?" Well, that is a separate talk in itself, but the main idea is you're breaking this down into parallel sub problems. For instance, this is a union of three problems, right? Who are the readers are, who the writers are, and who the admins are. And once this is traversed to check for Jill, you also know that Fred

is a reader and Kim is also a reader. So, say immediately after this Fred wants to read the document, that check is already done and it's stored in the cache. And this Google's Zanzibar system tries to aggressively store as much as possible So, Zanzibar came out in 2019, the paper, and this made shockwaves in the world of IAM, identity access and management, and you know, everyone was

like, "Wow, this is amazing." And turns out it actually works well when you have requirements for things like low latency, but a high throughput of authorization checks. You know, especially when there's lots of hierarchies of data and things are changing constantly like, you know, say documents or code repositories and so And turns out this is ideal for AI use cases because when you're working with AI agents

and rack pipelines and MCP servers, you have a high throughput of authorization checks. You want low latency cuz well, duh. And also, there are so many hierarchies of agents being able to access certain subsets of data and so on. So, any Zanzibar style reback system is ideal for authorization in AI use cases, including in agentic Now, of course, Zanzibar is a Google system, and unless you work

in Google, you can't use Zanzibar. But, after the white paper was launched, there were open source versions of Zanzibar. And one of them is SpiceDB. And this is the one the company I work for helps maintain, but I also help uh you know work on SpiceDB. And again, this is completely open source and companies like Netflix, Reddit, GitHub, IBM, etc. have contributed features to SpiceDB. Of course,

as people in tech, the most important metric for us is GitHub stars. So, there are about 6,600 GitHub stars right now on SpiceDB, but if you're generous, you know, I'm sure the project could always use some stars. And what is SpiceDB? Well, like I mentioned, it is completely open source and it is an implementation of Zanzibar that I just mentioned, which you can use in your AI

projects for Now, the name might suggest this, but SpiceDB is not a general-purpose database. So, don't use it as an RDBMS. The idea basically is you have your applications, your microservices, your AI agents, etc. SpiceDB is the centralized authorization service, and it's comprised of three things: your schema, the relationships that I mentioned earlier between the things in your system, and of course the graph that is computed

to check for permissions. And we do a deep dive to see how this or any Zanzibar system really works. If you're a developer, the thing that matters to you is that you can just make API or GRPC calls to figure out three things, right? You can ask for permissions by asking can a subject take an action on a resource. So, for instance, can an AI agent edit

This could also be can user Alice edit document two. You can also do lookups essentially. So, you can say be like, "Give me all the agents can that can edit document one." So, maybe you have multiple agents, or maybe you have multiple users that can edit the document one as well. A Zanzibar system that uses ReBAC does not differentiate between users, agents, or any objects. It's all

just relationships between different things in a system. And of course, you can do a reverse look up as well and say, "List all the documents that can uh that agent one can actually edit." You know, so it's a reverse look up. So, let's get into each of these and see how it applies to agentic security. And the first one we're going to talk about is schema. Now,

earlier I spoke about how in any Zanzibar-like system, you have to look at the different objects in your system like users, um AI agents, uh documents, code base, whatever, and then sort of define the relationships in between them and compute permissions based off of them. So again, going back to our very standard Google Docs example, let's build a system where you have documents and you have users

who can write to this document. So, they can view and write to the document. And you also have users who can just read the document again, like Google Docs. To build a schema for something like this, you would define it this way. So, your objects right now are user and document. Now, you have to define the relationship between them. In this case, a user can be a

writer or a reader. Writers can edit the document. So, that's a permission. Whereas, all the people Readers can view the document and also all the people who can edit the document can view the document. again, very much like Google Docs. So, you would define your schema this way and then you're like, "Hold on, my system also needs AI agents that can only view documents. How do I

do that? You know, do I have to break my entire system apart? Like I mentioned earlier, the cool thing about any ReBAC style or Zanzibar style all these objects, they're just different objects in your system. So, you can just define an AI agent and say, any user can read a document or any AI agent can read a document." So, very straightforward, you're just adding two lines of

code to add permissions for Once you define the schema, you have to add the relationships in between things. So, the relationships basically, you know, bind together these different objects. So, in this case, if you want to add a relationship for Jill to be a writer, you would write something like user Jill is a writer of document some document, and SpiceDB would create this sort of graph. What

about if you wanted to do it for an AI agent? Well, again, say you have an AI agent named Skynet. I know, I know. it doesn't differentiate between this, right? An AI agent is also a first-class citizen here. So, you would just write a new relationship called agent Skynet is also reader of some document. If suddenly you don't want Skynet to read some document, you would create

a new relationship where you would delete an old relationship or you would update a relationship cuz CRUD operations are possible, where you would remove this particular relationship, and then Skynet, the agent, won't be able to access some document. In this example, though, uh there is no relation again between the user and the agent. The user acts completely independent of the agent and vice versa, but maybe in

your system you're like, "Okay, this is fine, but I want my agent to act on behalf of the user. So, what do you do? Maybe your requirement is an AI agent can view a document only if a user can view a document. This indicates there's a relationship between the user and the agent. So, what you would do is add a relation between the agent and the user.

And in your view permission, you could this thing, right? So, the arrow indicates a sort of traversal where you're saying that an agent has a delegates permission. So, you move from reader to delegates. So, if a user can read a document uh the AI agent associated with the user can also read the document. We also know that if a user can write to a document, that they

can also read a document, and hence you add that writer delegates permission as Here, when you're creating relationships, you also create a relationship between the user, in this case Fred, and this document, but also the agent, in this case Skynet, and the user. So, you're linking the agent and I know this might sound like quite a bit, but I'm going to show you a demo which I

think will, you know, clear it up for you. So, the idea essentially is you first create this schema of different objects, you write the relationships, and then you can check for permissions. And again, this is a simple API call where you can do one of three things. Check for a permission like, "Oh, can read some document?" And you get a yes or a no. Or you can

do lookups to see what your agents or users can access. Or who can actually access, you know, um the document here. I'm going to show you a quick demo of this in action with an AI agent. But, the idea here is to add fine-grained permissions for AI agents or agentic security because what we're seeing in the ecosystem is AI agents being granted excessive functionality, but you'd rather

scope it to exactly the thing that the AI agent is supposed to do. Excessive permissions, for instance, maybe you add a library and the library has written over-scoped permissions and you run that library on your local system and an AI agent does something it's not supposed to. Or excessive autonomy as well. You can also add a manual approval workflow as part of an AI agent permission. So,

for this demo, basically I've built something called an agentic rag system. Rag, for those of you all who might not be familiar, stands for retrieval augmented generation. Uh it's a pattern we're seeing a lot in production by enterprises now where, say you have a large corpus of documents, you know, 50,000 documents and you want to add like say a chatbot to access information from those documents, you

essentially augment the knowledge of an existing LLM with knowledge from that rag pipeline. Here's the thing. I need to be able to get information only based on the documents I have access to. Say I'm my internal rag chatbot, I ask the question, "How much money does my CEO make every year?" I should not be able to access that information. And for that, we need fine-grained permissions, which

is the demo that I'm actually going to show you So, I'm just going to open that up real quick. The demo is on the GitHub link that you see there. And [snorts] I've added a front end for this as well. Right, so this is an agentic rag authorization demo. Now, how this works is there are 50 different documents, which actually maybe I can just show you those

documents here. This is the data. And you can actually see 50 different documents. Lots of documents about finance and engineering, etc. And we have complicated permissions around this. Right? These permissions can be complicated because there are different users. Uh and there are different departments and there are different hierarchies. So, some documents belong to some users, some to departments, some are public, And let's see how this works.

So, we are going to try an example of Alice in engineering asking about architecture. So, I'm going to submit this query. And this is live demo. It's using the OpenAI API. It's running against a SpiceDB uh permission system that I have running on my local machine. And it's retrieved five documents out of which, because Alice is in engineering, all five are authorized and you get an answer.

But here's the thing. Let's change the user to Bob from sales. Now, sales does not have access to all of these documents because, you know, sales might not have access to some engineering docs. And in this case, five documents are retrieved, but only one is authorized. So, Bob gets an only based on the document that he is allowed to view, which in this case is Similarly, you

I can ask I can do the reverse. I'll change it to Alice and ask uh sales question. Let's see how that goes. And again, maybe Alice has access. Nope. Alice has access just to the one document, and that's the one that Alice gets access from. These are documents that Alice does not I'm going to try a general question saying, "What is our company value?" Now, this is

a document that's open to all, and let's see if that works. Yep. Authorized three because the company handbooks and the code of conduct is open to all, right? And you get the answer here. Now, how this works is if I just open up my Visual Studio Code is I've actually defined different nodes using the LangGraph's library. In fact, there's a LangChain LangGraph integration with SpiceDB that I

wrote, so I'm quite proud of it. Where there are different nodes. The first one retrieves from the vector database and then this is the important part. You're checking for authorization for each of the documents that are retrieved and then the AI agent does a reasoning about whether it should look again, whether it should you know, get more information and then it uses the OpenAI API for generation

generation of answers, which you saw. Let me show you the diagram of how that would look. Yeah, so this is called post-filter authorization. So you have a user and an agent query which goes through an embedding model. That embedded query with a filter goes through the vector database. And in this case, we used Weaviate as a vector And you get the relevant answers back and this is

the key. There are metadata to show which embeddings came from which document and you can use that to check to see if this user has permission for each of those documents. And this is what SpiceDB is actually doing. And only the ones that the user has access to is then sent to the LLM to get a response back. So, the important part here is there is very

much a security boundary here to make sure that the LLM sees only documents that the user is authorized to see. And hence you're keeping your data safe. You can move this to the step here before the vector database, which is called a pre-filter authorization, in which case instead of checking for permissions on each, you can use a lookup resources call like the one I mentioned here. To

be like, give me all the documents this user has access to. And then you use that as a filter. And again, the LLM If you're wondering if this sort of um pattern is, you know, if is it being it's new, so is it being running uh in production anywhere by companies? Well, I'm assuming you've used ChatGPT. If you've clicked on that little plus button there, you can

actually connect ChatGPT to Google Drive, OneDrive, SharePoint, and so on. And this used to be called ChatGPT Connectors, it's now called ChatGPT Apps, but ChatGPT Apps actually uses SpiceDB and relationship-based access control at scale. Um this was many months ago, uh but at that time it was 37 billion plus documents across 5 million users. I'm sure that number has increased. So, this pattern is being used in

production by like top AI companies such as OpenAI. So, if you're building with AI agents or MCP or RAG, do look at using some sort of relationship-based access control or a Zanzibar system to make sure that you're scoping your agents, your documents to only what it's supposed to access, and you know, that could potentially reduce, you know, any data leakages or any security threats at at that

can happen at any time. The one key thing that I want to mention is this authorization node is not should not be skippable, right? This is a anti-pattern that I see sometimes. Do not get your AI agent to reason if it needs to do authorization. I hope that makes sense. Don't get your authorization. You have to have to have to mandate it. And the reason is this

is how prompt injection can work. Because people can figure out that edge case where you can skip authorization and get access to data it's not supposed to access. So again, make sure that your whenever your AI agent is reaching into any documents or code or whatever, make sure that this authorization check actually goes through. All right. Um I hope that was enough inspiration and I hope I

scared you a little bit about AI agent authorization and why it's so important. So next steps, do check out the Zanzibar paper and this particular demo. Again, SpiceDB is completely open source, so you can use it in production. I know large enterprises that actually do that, so do check it out. If you're using LangChain, I wrote this LangChain SpiceDB library for easy integration, so check that out.

And feel free to learn more from YouTube as well. I'd be happy to connect with you on LinkedIn if you have any questions or want to chat about something cool that you're building. I hope you enjoyed this talk. I hope you learned something new. Enjoy the rest of the conference and see you soon. Thanks and bye. >> [music]