Tailor Made: Dynamic Fine-Grained Authorization for API Traffic - Erica Hughberg & Andres Aguiar

This talk emphasizes the importance of real-time authorization in managing API traffic, particularly within the fintech sector. The speaker discusses challenges associated with traditional authorization methods like OAuth, which often require waiting for token expirations to revoke access. By integrating Envoy AI Gateway with Open FGA, they aim to implement flexible, scalable, and immediate authorization checks. The presentation also includes a demonstration of updating access permissions in real time by modifying a graph-based authorization model. The speakers highlight the seamless interaction of these technologies and their potential to simplify complex authorization processes, advocating for creative solutions to access management challenges.

And this is uh to me quite exciting because for me this presentation is putting two of my most wonderful things together because I happen to have dealt with a lot of API traffic in my career and uh I thought security and authorization was one of the hardest thing ever which is kind of the opposite to you maybe. >> I don't know I tried to been trying to

fix that for a while. I don't know if we succeeded but keep trying. >> Yeah. And I remember when I got my head around OAF, it still had some challenges that it wasn't really solving for me because for example, I worked in fintech for a while and people were like, well, if we remove access to a portfolio, we want them to not have access to the portfolio

straight away. And I'm like, ah, okay. So, I can't really bake that in to the token because if I did, then we have to wait for the token to expire. We have to have like back. It was a lot of mess. >> And the good news is that you see your boss asks you this. Wonderful question. >> So, and is your boss texting through iMessage? >> Interesting.

>> That's got a Slack message overnight, but not that iMessage. >> And now, you know, when it's important, you get iMessage. >> Yeah. If you work in fintech, fintech is more pressing, I guess. Yeah. >> It's like got to get it sorted now. So the challenge here that I've had to deal with multiple times is want to both protect internal resources also as well as external because

sometimes we route API traffic to resources we do not control. Um but we want to protect those internal apps and we make those this is the hard part make the authorization changes take effect immediately. >> Right. >> looks hard >> looks hard. I I think I think you might have a solution up your back pocket, Andress. I think we're gonna be fine. You You forgot it. You

didn't bring it. Andreas, I told you to prepare. >> You should make a ticket. >> Make a ticket. Yeah, cuz he >> and I message to someone to get ready. >> So, what are we really dealing with here? What does the environment look like? So, the thing is the first time I dealt with this problem, there were no agents hanging around. They've come along now, but actually

they're not that awfully much different. But we had a lot of application like server side apps making requests. >> I wonder why we keep using that kind of smiley thing for Asians because they're looking more scary. >> You think they're scary? >> Right. So they can do crazy things. I don't know. We need to change. >> They're a bit naughty. I think they might need a bit

naughty face. Yeah. But they look they look like they're a bit happy though. They're a bit approachable. >> They might be happy. Yeah. We are happy. And then so where are these clients as I'd like to call them sending traffic to? They are sending it to the old and trusted API services. Uh and now we also sending stuff to tools and we are sending things to other

agents. Very exciting. Now they're talking to each other. Look, that's why they're so >> That's Maltbot, right? That's when the agents talk to each other. >> Yeah, but I mean they're have the Maltbot. Yeah. They're on their own social media. Yeah. >> They don't need us to chat anymore. They're having a great time. They don't need an API gateway in the middle to chat anymore, right? >>

Yeah. They're just chatting with each other. So Andreas, we have a lot of requirements here. Do you what do you think about that? >> So when we talk about authorization, these are the things we care, right? It needs to be contextual, right? So who, what, when, in what circumstances, like what time of the day, from which I for which resources. Uh changes need to happen in real

time because if not Erica's previous boss, I want to be happy, right? I'm probably not my current either to be fair. >> And uh the other thing is I I it's not that important if you can just read documents, right? I want to know can you read this specific document or access this specific account and uh if things don't work and I want to remove access, I

want to do that immediately without waiting for a token to expire and get another one. And and then I want to know what happened, right? So who access what and when? If you want to implement authorization for users and also for agents, it's becoming more important. Yeah. You want to know all of those things and be able to do that. >> Like did we actually succeed in

applying those revocations for example like so the audit is really important. Like if you ever dealt with auditors you know exactly that it's important that you pay attention to these things. So basically we end up asking these questions don't we? like how do I authorize on the fly? >> This is kind of if I Google this, these are the suggestions that I get. >> Yeah. Yeah. Because

he's like trained his Google to answer his own questions. >> They all talk about authorization, but like it needs to be fast as well because one of the challenges that I had, right? So I own gateways and so people be like, "Erica, can we do more authorization on the gateway?" And I'm like, "If I do, it's going to be slow. Are you happy with that?" Or you

can just do that check closer to your service and I just do the you know first check is the you know is the token valid is it the right audience you know those things >> and um the whole team access thing and I'm like should I have to deal with that on the gateway I felt like they were pushing work to me they didn't want to work

did they just want me to do all the work so you know this is my favorite architecture diagram Andress my favorite one because we're all drawn >> This works for everything, right? >> Everything. This is what this is what we do. Don't don't over complicate your work. This is literally what you're dealing with. You have something that's calling something else. >> Very simple. No, but then obviously

we know it gets more complicated as we move. >> We make it complicated, right? >> You know what? Sometimes I think that people do think that platform engineers and you know, infrastructure engineers are just making things more complicated. >> Yeah, we are. Yeah. So that that's complicated, but to bring our friends back just to remind you what's where. >> Do you remember? >> They're still happy. The

Asians. Yeah. >> Yeah. They're still happy. Yep. >> They have free access there. They can chat however much they want. >> Yeah. That's why they are free. They're happy, right? Yeah. >> So, as we said, I wanted it to be have a flexible and fast gateway that controls the traffic. I'm I'm quite good with >> Yeah. Also if we want to make authorization work for with these

conditions needs to be again fast scalable. So it you kind of have to make three API calls to different services to know if the user can access this specific resource right you want to do something that is very quickly and uh they can scale and those kind of things >> and for some reason a lot of people are like Erica make it fast then that's actually how

I met Andress once upon a >> like a few years ago stayed over this >> that was the beginning of a beautiful friendship >> beautiful friendship. So if you if you want to know um save the date for the big day uh no wonderful uh marriage made in heaven between envoy and open FGI and >> I don't I don't know if you noticed but yeah she put

a lot of effort in in the slides and every time so these are the colors right so this is envoy color and the open FJ color you're going to see the green and the pink interact through the slides in very interesting ways so pay attention to that I'm very happy all of that effort is not lost, right? >> But uh you know what we talked about, right?

Like it's very nice to do that coarse grain stuff on the gateway. I'm happy to do that work. >> Yeah. So and course grain is the token valid has the right audience has the right claims, right? You can do that in the gateway. It's pretty easy. But in general what we do is we do that in the gateway and then when we want to know can this

request can this user access this resource we usually do that in the application side right? So we implement that in our own applications are the ones deciding for fine grain permission. So coar grain in the gateway fine grain in the application that's the way we always did it. >> Exactly. And and because I am a big fan of authorization we actually made it super easy in on

gateway like with a kubernetes uh cd there that you can actually define the coarse grain stuff super easy but it does not solve the fine grain problem because it gets static and yeah so what are we doing? Can we do both at the gateway? Almost. I >> let's see if we can do both at the >> So, what's really nice about Envoy is that if you have

not been in the Envoy space over the last year, something that's really exciting is called the dynamic modules. Envoy has always been very extensible, but dynamic modules makes that even easier. So, we are able to then very easily introduce a new filter to interact with Open FJ. >> That's pretty cool. And how how you do >> secrets? I I know. I mean like I'm not going to

tell you. No, I will tell you. >> Okay. Okay. So, you tell us. Okay. Let's show it first. >> So, the important part here is we're going to show you this as a proof of concept just to make sure that we all understand that this is very possible and the opportunities it creates. We want to go the right way because otherwise we're not Where is my demo?

That's a good thing to find. Right. It's over here. Fantastic. There we are. Um and it's probably a bit small. So, put your glasses on and find your binoculars because that's probably needed. So I know that graph based or openfj actually it's a graph isn't it and >> yeah you can define a graph when data and a model that it will help you navigate that that graph

to make authorization decisions. So we're going to see it in action now. >> Yeah. So we have some people here Alice and Dave they are part of the engineering team and then we got Carol part of uh no actually Alice is in engineering. Dave and Carol are in product and we have some stuff they want to access. And then Alice, she's got a little code assistant there

as well because she's got the little agent helping her out which uh you know we all want. So let's just talk about if we're going to check the user access here. So Alice can access some cool self-hosted model. That is what we call our fake mock model in envoy AI gateway when you get that quick start up and running. So very cool. But as we can see

here uh Alice can access it because she is part of engineering. I'm going to put that in space. As you can see, Alice can access it because she's part of engineering. Carol cannot use it. Oh, and um Dave, he can use it. I'll tell you why Dave can use it. He actually if you use your eyes very carefully. There is a green line from Dave to some

cool self-hosted model. He has direct access. He doesn't have it via team permissions. >> So Dave is kind of special. Has the permission on that model by himself. It's very special. >> He got some bypassing team permissions >> Very special. No, but that happens, right? We have scenarios where specific people needs different specific access. >> So, we're going to hire Bob. So, now we've hired Bob. He's

a contract and has joined our engineering team. >> Bob the contract the builder. Bob the contract Bob. Bob the builder. Bob the software builder. Yeah. So Bob has joined the team and uh we are not gone and actually added a bunch of permissions to Bob. Bob just joined the team. So if we check the user access again, we can see that Bob is now able to access

some cool self-hosted model. When he was not employed and we did not pay for his services, he couldn't. Um so that is Bob. But then we have this agent here. It was nice to know about the agent. The agent actually belongs to the engineering team as well. So if we check the agent access the agent can not create issues in GitHub but it can read them and

get them but if we give uh see if we so we want to grant engineering team right access so now we've added another resource there. By the way I just want to be very clear we are updating the open FGA graph in real time here. This is this edits the graph when I remove it and add it. It edits the graph that is powering the >> and

those four milliseconds there is the time it takes to go through the gateway open everything that's going on. Right. >> So so these times you see 4 milliseconds that's the round trip to the UI. >> Right. Obviously this is running on my machine but that really shows you that the speed inside of the Kubernetes cluster can get very fast. So it doesn't like many of my colleagues

used to be scared of Erica if you're going to add all of that to the gateway. It's going to go real slow. It doesn't. So if we check the agent access again, it's been updated. And then we can just remove it. And you can see that it's instantly propagated. And uh if we check the tool access, similarly different people can access different tools depending on their relationship

with them. So, if we come back to my presentation that is somewhere over here. There we go. What is a You may be like, "Okay, Erica, that was a cute little UI you click buttons in." There's actually an architecture behind it and it may feel very simplified, but I do like simple boxes and arrows to be fair. We have a demo app that calls envoy AI gateway

and I think Andress you and I talked about that there's actually quite a bit that happens in envoy AI gateway there to give context to open >> and what is that composer dynamic module. >> Yes. So that is something to make it easier that we have in built on envoy is a opensource solution easy for me to forget about talk about that but if you go to

built on envoy.io IO uh you can easily build extensions to envoy using dynamic modules but composer is a specific type of module that makes it easier for you to put in go filters. So I built an open FGA um module and it was really exciting is I have my PR open in the open source there. So uh it's been a busy week but we'll get that in

there. Yeah, the goal was to merge it before the call, but there's Erica is too busy. This week has I don't know a lot of things every day. It was hard. >> It's been busy. >> Yeah, it's been busy. >> But we will be there. My PR is there. If you want to take the code, you can go take it. >> You can review the PR too.

>> You can review the PR. Yes. But but don't don't be too harsh because we want you to get in. Uh so there we are. That was uh that will make it very easy for people to use this together with Envoy Gateway, Envoy AI gateway or even Envoy proxy like in a standalone mode or in a different scenario. as long as you aren't running on web proxy

and um the open FGA deployment I made I made it very looking simple there but >> yeah it's a simple model but the way it works is you seen that the demo app is writing to open FGA so when permissions change open FGA is a permission database so we need to update the permissions in the database right so that demo app is doing that and then the

gateway is just reading from there checking if the user has access to the specific resource in this case is the model right could be whatever resource your application is about. >> So it's a it's and it's so fast like something that I really like about Open FJ is like when you do write the update is really fast >> and because solutions that I tried to build before

I came across Open FJ was like yeah I was using databases myself. I even been in a situation where I used graphs. I just remembered this is real time Erica remembering that my first ever hackathon I participated in my corporate career which is over 15 years ago was creating a graph-based system in Java. >> Huh cool >> which was not as fast as your one but there

we are. Yeah, go is it just is go is faster than Java maybe >> and maybe architecture uh comes into play but I think that's really interesting but you know what we said we showed a very simple thing and to be >> yeah so I'll introduce very quickly how how open FGA works so in open FGA you what is in the left is what we call an

authorization model and here you declare the entities that are relevant when you're making an authorization decision and you can model anything here right in this case it's modeling something uh uh the use case is simple which is we have team agents and users that have access to model tools and resources and we can assign direct access to the user to the members of a team or to

the agent to any specific model tool or resource so that's what the model try to express and then you instantiate that model with data we call those relationship tupils and are the form like a user Alice is a member of the engineering team the user Carl is a member of same team user Dave is a member of marketing team and then the members of engineering team can

use specific model members of engineering team can use another model Alice can use that model directly the code assistant can invoke the specific get issued and the permission action right so you can write the data to instantiate the model to manage to configure the permissions in any way you want and then in the gateway you are going to ask questions like this can user Alice use model

GB TB4 as Ali is a member of engineering and the members of engineering can use the model then she can do it that is a very simplistic way can get way more >> and I think for me what was so exciting about using open FJ was to have that flexibility and it opens up so much creativity to create solutions to problems I was facing. I think one

of the very common challenges we are actually dealing with in envoy AI gateway is this check of teams accessing certain models because a lot or even providers so for example like which teams have access to what models and what providers etc. And that's a really real application for enterprise uh users. So I I I really like graphs if you haven't understood that yet. Um, so how did

we open how do we access it? We talked about that uh dynamic module that was created, but it might feel like really overwhelming to be like, "Oh, it's going to be super complicated, right? It's going to be so much code, so much configuration to create these rules." Good news is using the extensibility of Envoy also able to create a extensible way of configuring this. you're not seeing

the full configuration, but if you do go to the built-in envoy site and the repo, you will then or especially the repo, you'll be able to see how you can configure this. But the only thing above there is like which type of uh resource it is. But basically, we saying load the plugin. We're going to and we're going to configure Open FGI and we're going to set

these uh rules up and important we are failing closed. So if we can't connect to Open FJ, it's actually going to reject the requests. And here's just some of the rules I created. But because we are using Envoy AI gateway, Envoy AI gateway has filters before that populates the AI EG model. So that allows us to uh put model able to use the model in the open

AI API open AI request for example. >> Yeah. So what is interesting here is that to to call FG open F M F M F M F M F M F M F M F M F M F MGA you need to know the the identity of the user the action it wants to perform and identity of the resource right and that data in general is somehow

in in the in the request right but in addition of the data of the request in envoy you can populate a lot of metadata in previous filters that you can also use that when making a check request and the scenario we were discussing was uh what if you want to model quotas right a specific user can only use this number of tokens for this model. >> And

actually there's the there's a an >> a rate limit service >> a rate limit service that gives you that information in the context of the call. So you know how many tokens it consumed >> you can you can store in FGA what is their quota right so you can compare in the check request if the user has permission to access the model and is in the quota

limits that it needs to have. So what's kind of cool then is if you have you know premium and basic for example and you can instantly update Erica's access from basic to premium and because we instantly changed it in the graph it means that next time we check if Erica is within her limits she is because now we updated it instantly. So that's really exciting. There's loads

of interesting ways you can think and I what I really want people to leave here with will be that this is a tool that you can use to creatively assert business uh rules in different ways. It's not just about identity or team membership. It it goes beyond that. >> So did we achie do you think with this P have we will we be able to do all

of these things? >> Yeah. So we're using the model the user user is using we're getting it from from the other filter which is great. It's dynamic. We show that we change things in real time fine grain. We went to specific model. It's not can I use a model right? Can I use this specific model? We got access immediately and uh and we didn't see the audit

part right. >> Yes. We didn't talk about that. >> Yeah, you're right. But I mean we all know I guess that Envoy has a very good >> loing capability. So you can see all the everything that's happening there and Open FBA also has that. So in open mga will be this user tried to access this resource that's the data you have in invo you have all the

data from the request. So it's >> and you can choose how much you're populating how much you're logging. So you may want to tweak that for your needs to be the logs that you want but also integration with different logging capture systems. You can even get it into data dog if that's something you enjoy. Um so what is this really about? It's about being stronger together because

combining these solutions helps us achieve more than what we would ever be able to achieve as one like if I was trying to build an open FGA type solutions inside of one myself that would take a lot of time and a lot of effort. >> Yeah. And and the other piece that which I think is interesting is that in the past we couldn't do uh fine grain

access in the in the gateway because you need data to make those fine grain decisions, right? So that data about your resource, your permissions is in databases. What Open FJ does is put that data in a single place, right? So now the request is is very fast and given you have all the information in the request to decide what to ask for. >> It's a great solution

to put in the gateway, right? There's no other data access that you need to do. you just call up an FGA and you you know that >> and I think for me it's been so nice to be able to be creative with this and solve real access needs and I if you are happy to chat about this by the way feel free to come back by either

like envoy or open FGA uh kiosk so if we yeah literally was talking about imagine what you can do with this combination and they are fully open source there is >> yeah when we were preparing is we were think oh we should do this and we should do that we should add this and we okay let's calm down and add something simpler and so we can finish

it on time >> yes >> and then we can get more creative >> I unfortunately it's very is a lot to say in this space and what we can build but yeah please get involved in our communities and we have some QR codes there which uh because we thought writing down big links was not going to make it easy for you. >> Yeah. So the one with

the open FJ thing is the one for open FJ and the one without is envoy >> on the pink the colors the colors keep keep making sense in most cases >> pink envoy green and blue open FGI but this link will take you to community for envoy it takes you to community inviter for the slack the reason I did that is please join our community you can

find out more about envoy proxy envoy gateway envoy AI gateway there and uh I host the envoy AI gateway community meeting every Monday well when I'm there otherwise I want else host it. But I'm I'm there quite a lot. You got some community meetings as well, I believe, that people are monthly for now. Not every Monday, but yeah. >> Yeah. Well, that sounds like a lot less

work. Maybe I should No, it's joking. Should reduce the number. But who are we? Because you probably don't care who we are. This is who we are. And we should swap places. No joke. >> Yeah. Yeah. Right. Yeah. Because everything is fine. But >> no, now I flipped the sides, didn't I? >> Yeah. But >> I need to look that. I'm Erica Hubber and I work at

Tetrate. I'm an Envoy AI gateway maintainer and also really involved in the Envoy community as a whole and I'm really excited about the work we are doing together and also now together with you. >> Yeah, I'm Andreas. I work at Octa and I maintain part of the team that maintains Open FGA >> and now now I need to go and touch this >> Okay. Yeah. Yeah. There

we go. Nice vist. >> Perfect. So, thank you very much. We're going to be in our own fields conducting