Open Policy Agent. (OPA) Intro & Deep Dive - Charlie Egan & Anders Eknert, Apple

This talk provides an update on the Open Policy Agent (OPA) and its associated projects, highlighting the importance of OPA as a general-purpose policy engine. The speakers, experienced maintainers of OPA, discuss recent developments in the Rego language, focusing on new features like string interpolation and the introduction of the 'or' keyword. Additionally, they share insights from a community survey that reveal growing adoption of OPA in production environments and improvements to user documentation. The speakers emphasize the significance of community engagement in spreading the adoption of OPA and encourage users to contribute back to the project. They also introduce tools such as Regal, which aids in Rego development, and discuss upcoming enhancements to OPA's ecosystem.

Awesome. We have a full house and some people were couldn't even make it in. The gatekeeper is effective. They'll have to watch online later. Yeah, thanks for coming everybody. It's um yeah, a privilege to have a full room. Um yeah, thanks for coming. Um so yeah, this is the open maintainer track. Uh this is uh going to be our update for for this event. Um uh first

a few uh short little introductions about ourselves. I'll let you go first Anders. >> Yeah. Uh I'm sure many of you have seen me here before. This is my sixth or seventh CubeCon. So uh and I have been involved in the OPA project for for just as many years. Uh I started out with identity and access control. That kind of brought me into OPA. And uh so

I started out as an end user uh and eventually started contributing and uh for the past five six years I've been a maintainer of of Opa and a few adjacent projects. >> Yeah. And so hello everybody. I'm Charlie. I first used Oper as a user in 2019 for a Kubernetes admission use case and have used it both in a a product uh since then as well. I

was working on a a product which did policy around X509 configuration in Kubernetes environments and um was enjoying using open and contributing to the project working with Anders and team. So I I joined uh joined Starra at the time at the end of 2022 and I've been a maintainer of the project since then as well. Uh so yeah I was also a user a user first. Um

so hopefully it's a little bit of inspiration for some of you users if you're interested in getting involved in open source like being a user is a is a great place to get started with our project and all the projects here really. So yeah, um today we're going to talk about uh for those of you who are very new to OPER, a quick introduction about what our

project is, what it does. Uh we're going to cover some updates from the community survey. Just quick show of hands. Did anybody complete the community survey we ran at the end of last year? One or two. Yeah, thank you. It's appreciated. Um and uh yeah, then we're going to go over some recent updates to the to the Rego language and some some changes that are upcoming as

well as um yeah, other things that are next for our project. So um so yeah, uh Opra's a general purpose policy engine. Uh you uh you might not know what that means. You may not be familiar with the idea of a general purpose policy engine. What can you do with a general purpose policy engine to begin with? uh you you haven't used one of them perhaps, but

you've certainly written some policy functionality uh in in various different languages. Uh whether a particular runner can deploy to a particular environment, what users in a a given system are permitted to do or who's allowed to make changes in a particular uh git repo. Uh and you might have written uh these policies in all sorts of different languages depending on where you wrote them and what they

were for. Um, and that's what our project is about really. It's it's giving you a standardized way to express policies for different use cases. Uh, not just authorization, not just Kubernetes admission, not just controls around uh source uh source control and file updates, but uh one one policy language for all these different use cases. So um yeah and the idea is that you know you're you work

on you work on one team but OPA has value when it's used by lots of different teams who previously implemented often very similar policies in lots of different languages maybe a little bit inconsistently and that's what we we're hoping to offer our project. So yeah, you use oper by writing some rego. You load it into an instance of open policy agent and you provide structured data about

the decision that you would like to make uh typically as JSON and OPER will respond with a JSON response. At the same time, Oper is also responsible for reloading your policy as it changes, as you make updates to it, as other teams make updates to it. uh as well as producing audit logs about all of the decisions that have been made whether a user was allowed access

or not or whether a resource was um you know denied from being created or not. Uh and so when we talk about general purpose policy engine that's what we're talking about. So just a a quick little overview now about um some some data that we've we've learned about how people use the project and things that people are doing with Anders found us found a study from earlier

in the in earlier this year. Um there was a study done about different policy languages that were being used to express uh express policies found on GitHub. This was published in January. Um this is uh noteworthy or it's noteworthy because uh often policy is a security uh some security code is not typically something that people do publish but of the policies that are being published uh a

lot of people are sharing their oper code and this was uh something that this uh this paper found and it's exciting to see that that um yeah of the code that shared a significant amount of it is written in Rego and that's exciting for us because uh we spend a lot of time thinking about Rego and trying to make it work well and make sure it works

well for different use cases and different users, different levels of expertise. So, um yeah, that's great. Uh is anybody sharing any of their Rego code on on GitHub or on another public public space? Again, handful of hands, but yeah, that's great. If you're able to do it, that's brilliant. Um so, yeah, some other stats, some quick stats from the community survey at the end of 2025, end

of last year. Um we won't go into all the details. the the full results are on the OpE website if you're interested to see them. But this is an important one I think is is worth going over. Uh just showing that this is based on the question what's your most advanced use case for OpE? Uh it's uh increasingly most people's uh advanced most advanced use case of

Oprah is in production. Uh that's something that's changed significantly in the last five years. Hopefully the trend continues. Um I think that's an important one to share. And yeah, I think another important one to share is like we're here in person, we've got the talk, you're all sitting here. Um, we've got the kiosk in the project pavilion. Uh, when we asked about how people first came across

Oprah, it tends to be that they hear about it from their colleagues or they hear about it by coming to an event. Uh, and that's that's significant as well in in an increasingly disconnected world. It's um yeah, it's great to see that that the project is spreading via word of mouth and with human connections. That's that's important. And I think, you know, I would like to, you

know, ask, you know, who could you tell about Oprah? Who could you tell about what you're doing with it? Um, you know, what could you do to to help us out here? It would be appreciated. So, yeah, I'm going to hand over to Anders now, uh, for a few quick updates about, uh, recent updates to the regular language and some other things that are upcoming soon. >>

Very nice. >> I got to do the the nerdy stuff, so really thankful for that. Uh, let's jump right in. Uh so I think if you haven't used Rego for a few years, please try again. There's a lot of things that happened in recent years. Uh both in terms of like the language itself, but also the tooling and all all the the various things around the development

process. Uh the big one is of course this jump from uh Rego version one to Rego version two. That's that was in December, not last year, but the year before. So, uh, a year and a half almost since that. But we still we still see a lot of like older style Rego out there. You got a a whole bunch of new uh nice keywords that make it

that will make it more readable and easier to maintain over time like like uh the contains keyword or uh or how you can use like not in and so on to to describe policy in a way that that it's easier easier to map to how you describe it yourself. So uh check that out. More recently though uh we have or the most I think the most recent

like major feature is that of string interpolation uh and I'm sure many of you are familiar with string interpolation from other languages and that was something we have wanted for a long time in rego uh previously you you did normally use the sprint f function or concat or a few others but commonly the sprint f function uh and why would why would you need that Uh the

sprint f function is is very versatile just like rego and also just like rego it's it can be easy to get a little lost. Uh we needed something or we wanted something that was uh just as versatile but just as we have tried to move uh rego in a direction of being more like uh easier to use and and friendly we wanted a built-in to kind of

follow that direction. uh and I'm not sure if the animations here. Okay. So, so basically uh the first example you'd have a simple uh or what the examples here show are kind of all these kind of uh the versatility of the sprint f command but also kind of some of the quirks and complexities. Uh another another reason why we have wanted why why this is problematic for

us is that we're that Reag is now starting to target a whole bunch of uh different platforms like we had web assembly in the past. Now there are people are writing interpreters for uh for Rust for C++ uh for Java and what more and and sprintf is notoriously hard to get right in all these different languages and some things like the uppercase T here is like it's

it's basically one to one mapping from Golang and so you probably don't want that in Java or >> and I think like you know it's it's also for these alter alternative implementations, but it's it's also like this is a a relatively advanced feature which surfaces very quickly for new users to the language when um you know it might be possible to we think uh with our new

feature offer an easier way for those sort of first time that people are using Rego if they all of a sudden have to get up to up to speed on Sprint F. It seems like an advanced feature that appears very early in people's journey with using the language >> and 81% of all users are people who responded to so pretty much none of you in here but

our other users said they are you mostly using languages that have string interpolation. So making it easier to transition from uh one language to Rego. So what does it look like? Well, it's something like this. Uh, as you see, deny contains. And now you see a dollar sign because all good things in life has a dollar sign in front of him. And now you can basically just

inject a direct reference to instead of instead of doing this sprint f and and then have like placeholders, you can just inject the values directly. And we support even though it doesn't show here, you can you can even do function calls or like more advanced expressions inside of template strings. So you could say to lower or or whatnot if you wanted to format the the value inside

of the string too. So it's it's basically a full-blown uh expression evaluation. And our tooling supports this fully as well. We can do linting inside of template strings. We can do formatting. Uh you can do like >> go to definition etc. >> Exactly. Next up is uh the or keyword. I wrote a blog on this for a few years ago. It's called like how to express ore

in Rego. It's still the most read blog on OPA and Rego uh out there. So this has definitely been like a pain point for many. And even though I think we have like seven or eight ways to express or there's no way to just say or. So we were thinking like hey what if we just did that. And uh that is uh something that we uh intend

to work on uh in the next few months and hopefully we'll have something to show you all uh soon. Uh so yeah just just a few ideas about what it what what it would look like. Uh as you might know rules in Rego when you have when you have more than one rule of the same name those rules are implicitly ored. So if one of the allow

rules is true the allow rule itself is true even if the even if one of them or all all of the rest are not true. So if if we have an employee and is trying to read something and that's if both of those conditions are true, we say that's the allow the request is allowed. So uh what the or or keyword will do is just a way

to simplify that. So you can express uh the same logic but inside of a single and uh which which one to choose is of course that's going to be up to you. I think the the first way of doing it is is not going anywhere. That's kind of fundamental to Rego, but we like providing options and about options and learning uh new things. Over to you Charlie.

>> Yeah, thank you. Um so yeah uh on the topic of things that people have talked about or given us feedback on in the past uh there you know go back we went back uh to the previous user surveys when we ran the last one and there was a lot of comments about improvements to examples and something we've known about some time in our documentation. So uh

yeah and pe people like examples uh and uh so do the generative AI tools as well. Uh but crucially this is for for human beings or human beings first feature. Uh we released a new website last uh last year. It has lots more interactive examples uh as well as specific examples for built-in functions and keywords. Um so we've we've uh we've done a lot of work uh

on making the examples for Rego built-ins and Rego keywords in the past 12 months. Um so yeah do do check that out if you've got new people on your team learning uh learning language or learning over oper then um do send them to to those resources. Um we uh yeah uh we also yeah so the new site as well is a great way to get involved in

in in the project. Uh it's hopefully a much improved contributor experience if you want to add some new documentation or add a new bit to the website. So uh many of you will have used it's based on that. Um so yeah uh we as part of the new website we have a feedback uh form at the bottom of every page. Encourage you to fill it out. Please

try and write more than three words. Um it's sometimes very hard to work out what's meant and we feel like we're maybe missing one or two words and then we could do something. But um but generally speaking people submit positive responses so that's really nice. uh it outnumbs negative ones about 3 to one and for each of the negative ones that we understand we've we've done something

almost immediately. So uh yeah like we're an open source project we don't have you know telemetry um tracking your eyeballs and things as you used it. So any feedback that we can get from you is really valuable uh for improving it in in the future. So yeah please do fill out those forms or or or do whatever open issues etc as well. Um something else that came

out of the community survey is that um and this was done in December. I think this number would be higher now. Uh using LLMs in some form to learn ReGo or to do Rego uh Rego things. Um that's noteworthy. I think um we you know we've Yeah. So we've seen that people are having some success. they're able to use these LLM tools to um uh review existing

projects and get a bit of an understanding about what policy is doing or they're able to kind of draft some rules and then edit them themselves. Uh we still see a lot of users complaining about hallucinated functions and language features. Uh when you know point back to the examples we shared about not being lots of policy code out there on GitHub or or on the internet for

for these models to be trained on. And that's like a problem that we face. Uh we're not JavaScript. Um and so yeah, this is something that we would we would be keen to improve. Um yeah, people also comment, you know, we've been working a lot on making the language better and LLMs often generate this outdated Rego, which we've been working so hard to get people to move

on from uh as well, which is disappointing, but um so yeah, and if you if you're reviewing uh Rego PR in your team and you see import Rego V1, that hasn't been required for quite some time. So if you see that it's a little bit of a red flag. It's like hm was this was this generated by cord code or something. So do check that. Um please

please do um >> it's the you're absolutely right. >> Absolutely right. Uh yes except the import v1 that's no longer needed. Um so so yeah uh we're hoping to make some improvements to our documentation to make it a bit easier for these tools to consume and consume correctly. Um, we want to make sure that our llinter and language server is available uh sort of natively as well,

but we also don't have a full picture yet about how people use generative AI to write rego. And so if you're interested in giving us feedback, it would be appreciated if you visited this issue and either thumbs up, thumbs down. Thumbs down means I'm not using uh generative AI to write Rego. Thumbs up means I am. and then leave a comment if you've got anything that you

think is interesting about your use case or your successes and failures. That would be a great um a great thing for us to learn Um and just yeah, just to remind people that OPA is a security tool predominantly. First and foremost, it's a security tool. You can use it for other policy based decisions as well. Um but it's there as a line of defense against your systems,

your data, your customers data, your family's data. Um so you know please bear that in mind and and remember that the most secure way to use our project is to learn about it and to use the best tools uh that are available to you uh which we've been hard at work creating for quite some So yeah and is going to talk through a few uh updates now

from uh s sub projects in our in the opera or um and that's our our last section. >> Thanks you. All right, so moving straight into the good stuff. Regal, how many of you are using Regal? Yeah, quite a few hands. For you who uh aren't or are not aware what it is, uh it was basically a way that we we thought we would improve like the

the general uh policy offering experience a few years back. We kept getting the same questions over and over again on the Opa Slack and like in the OPA kiosk here. Uh people running into very common problems. So we thought like hey what what if we could automate answering some of these problems and uh and so Regal was born as a llinter originally where we tried to say

like for for these very common issues people run into we'll we'll write a lint rule. So all you have to do is just run into that issue and uh our tool will tell you what what you did wrong and uh in a very friendly way and uh and how what you can do to fix it. Uh a few years later we have I think 109 L rules.

So we have a pretty good coverage and the llinter has also evolved into a full-blown language server uh with all that means a debugger and much more. So it's basically your uh friendly tool to guide you and assist you in Rego development. So if you if you ever do work with Rego and Opa, I suggest you check that out. It's going to help a lot. Uh and

so as for the updates, we now have full support for the IntelligJ editor as well or like the the suite of editors. Uh we have oh I lied before and said I said 109. So okay so we have are 107 lint rules now and a few more has been added recently. Uh more more better support for the language server protocol. Uh if you're not aware like the

language server protocol is what your editor speaks to to Regal to ask for things like uh syntax highlighting, go to definition, uh autocomplete and so And performance we use Regal to lint regal itself. Uh Regal is also written mostly in Rego. So it's a it's a there's a lot of Rego going on here. Uh but uh we use Regal. itself contains about 300 policy files and we

lint those 300 in I think 400 milliseconds. So uh it's a very performant uh tool especially for when like written in in a in a in a policy language. Uh and for the future uh we do have uh we're planning to add another language server feature called find references. So you can just click on a function and see all the places where it's used. Uh rename so

you can uh rename a rule and have all references updated in your project. And we're looking at things uh to better integrate uh the AI tools that people seem to be using. Uh other projects OCP is a new one that is part of the OPA family or OPA control plane. uh it's basically a way to provide uh a centralized control plane for open policy agent. Open policy

agent is a is a distributed component. OCP provides uh a way where you can say here are all my policies and have your all your opas uh go go there and find their policies and keep them up to date. So it's a way to manage policies at scale. And recent updates in that pro project include ABS AWS S3 data source support, intermediate representation and web assembly uh

build targets, configurable bundle revisions and coming next is configurable uh bundle build optimizations. Uh Gatekeeper and com tests also two uh projects in in our family. And uh the Opa Gatekeeper has done two releases since uh since the past CubeCon. Uh and two major features from that project is Gator Bench which is uh a new command you can use to benchmark uh your Gatekeeper policies and Gator

policy which helps you discover and install policies from uh the the public gatekeeper library. the comp test project uh which is which is basically a tool to run OPA on on files on your local disk uh and that in that project we have uh improvements to output formatting and pre-commit hooks. So, uh I think that is that is uh the update and all for us uh for

and we're going to leave it up to you now if you're a new user. Uh a few a little of a few uh uh tips would be like watch or visit the the Rego playground. It's a good tool for you to start hacking. Uh and then after that do check out the VS code opa extension. Uh and of course with regal I think it automatically installs regal

now. >> Yeah it should do >> should do. Uh and again intell works as well but VS code is still like the most tested option. So if you if you're looking for the the easy path it's probably VS Code right now. And we have a Slack channel or a Slack uh community that you're free to join. If you are a sporadic user, maybe you wrote a little

Rego uh a year ago or so, make sure to check out. There's a lot of things that happened since then in the language and and the tooling. So, be sure to uh to check that out and and have your OPA deployments updated. Uh that is of course important too. Uh if you're a regular user, consider doing what we do here. Spread the word. Again, this is how

how you most of you know about OPA. It's not from me or Charlie. It's from the colleague next to Uh and remember, we also have like an ecosystem page for on the OPA web where you can add your project if you do depend on OPA or if you use it in any capacity. So uh and you're more than welcome to add your project there or your product.

We also have an adopters file that you can use for for the same purpose. And uh and of course if you want to go one step further you're welcome to contribute just uh create an issue or reach us on Slack. I think that's it. >> That's it. Yeah. Thank you. >> Thanks a lot. Any questions? Are we doing questions? We have a karaoke mic up here, so

might >> All right. Uh we're we're happy to answer questions up here uh later if you want to. >> We also have the kiosk and we have a session tomorrow in the pavilion as >> presentation. >> We forgot that. Uh we'll you know where to find us.