WIT Happens: Exploring the Latest Evolution of the SPIFFE and... Noah Stride & Arndt Schwenkschuster

This talk explores the evolution of SPIFFE and SPIRE workload identity standards, presented by Noah Stride, the lead of machine and workload identity at Teleport, and Arn from DeFactor Security. The speakers provide a recap of the Secure Production Identity Framework for Everyone (SPIFFE), which defines standards for identifying software systems across dynamic environments, and the importance of workload identity in modern infrastructure. They discuss the newly introduced Workload Identity Token (WIT), aiming to combine the strengths of X.509 certificates and JWTs while overcoming their weaknesses. The WIT is a JOSE-based token that protects service-to-service communication with cryptographic binding to the caller's identity, enhancing security by ensuring proof of key possession.

Cool. Well, welcome to WIT Happens, exploring the latest evolution of the SPIFFE and SPIRE workload identity standards. Today, we'll be taking a brief recap of SPIFFE and then we'll be talking about the upcoming WIT SVID credential. I'm Noah Stride and I'm the lead of machine and workload identity at a vendor called Teleport. I'm also a member of the SPIFFE steering committee and a maintainer of the SPIFFE

specification. I'm joined here today by Arn. Hi, I'm Arn. I'm a software engineer and standards architect at DeFactor Security. Also, like Noah, I'm steering committee member and maintainer for SPIFFE and and I I'm also active at the OAuth and WIMSE working groups at the IETF. Cool. So, a quick refresher on SPIFFE. I'd be interested, kind of hands in the air, who has SPIFFE deployed in their organization

today? Great. Cool. So, SPIFFE, the Secure Production Identity Framework for Everyone. The official uh definition is a set of open source standards for securely identifying software systems in dynamic and heterogeneous environments, which is not easy to say quick. >> [laughter] >> Um there's two parts of that I'd like to zoom in on. The first is software systems. I think in common parlance today we'd say workloads, but

we're referring to running code running anywhere, right? We're talking about virtual machines, containers, serverless functions, uh like physical machines, right? The list goes on and on. The other part, dynamic and heterogeneous, which I might be pronouncing incorrectly. Um kind of to me refers to modern infrastructure, right? In modern infrastructure, lots of organizations were split across uh you know, on-prem, multiple cloud providers, and the dynamic part, you

know, workloads are constantly spooling up and down. Um legacy techniques for identifying workloads and infrastructure simply just don't scale, right? Now, SPIFFE is just a specification, it is not an implementation. The most commonly well-known sort of uh implementation will be SPIRE. So, this is a CNCF graduate project, but there's also other implementations from vendors available and such. So, what does SPIFFE define? SPIFFE defines many, many things.

Uh some implementations will only implement part of the spec. Some of them will implement all of the spec. For the purposes of today's conversation, the most interesting part to us is the SPIFFE ID and the SPIFFE verifiable identity document. So, the SPIFFE ID, this is really foundational to all parts of the SPIFFE specification. I think pretty much all of them make some reference to SPIFFE IDs. The

SPIFFE ID is a string which is going to uniquely and specifically identify our workload. It takes the form of a URI. So, we kind of the host part here refers to something called the trust domain. And then the path uh sort of identifies that workload specifically within the trust domain. So, in this case, we have the billing service maintained by the payments team in our European zone.

Now, a is used for authorization decisions. So, which workload should be able to call other workloads and access resources, and can also be recorded for auditing purposes. Now, we can't just send the one workload to another and expect them to trust we are who we say we This kind of enables all of the benefits that organizations like when they adopt the first thing is abstracting workload identity.

If you have Kubernetes, you have service accounts. If you have AWS, you have IAM. If you have GCP, you have GCP service accounts, right? And you a way to have one form of identity regardless of where a workload is running in your environment. is what enables also sort of off-the-shelf SDKs and tools to exist. And this is all without locking into a Yeah. Now, about WIMSE. WIMSE

stands for Workload Identity in Multi-Service Environments. It's a working group established at the IETF in March 2024. When workload identity started to become a thing, we were kind of talking about it for like 3 years now. In the beginning, we were thinking, does it belong to OAuth? Does it not? Where should Where should this have a place? And particularly in OAuth, you would have stuff like consent,

user experience that just doesn't apply to workload identity, one of the reasons I love it. >> [panting] >> Yeah. Um the reason this is an IETF is because it benefits from a lot of related security standards and knowledge. There's stuff like OAuth, I mentioned it before, SCIM, PKIX, JWT and JWK describe there all the algorithms, and there's a lot of knowledge that we can leverage leverage. So,

yeah. This is WIMSE working group in OAuth. Currently, we focus on architecture, securing service-to-service traffic, and also document existing workload identity patterns. For example, instance metadata at cloud providers. Um sometimes you would get ID token in CI/CD systems, Yeah. So, we kind of try to document all of those, bring it together. They all kind of look similar, but they are yet different. So, yeah. Just documenting best

practices. Cool. So, before we talk about the WIT, I think it's useful to talk about the JWT and the X.509 SVID to understand why we felt a third type of credential was necessary. We'll start off with the X.509 SVID. So, this is obviously based on the ubiquitous X.509 standard, you know, the one that's identifying servers to us when we're browsing the web day-to-day. Now, an X.509 certificate

contains some identity attributes and the public part of a key pair that belongs to the holder of that certificate. We say that it binds identity attributes to a key pair. those identity attributes are going to vary depending on how you're using X.509, but in the case of an X.509 SVID, the identity attribute we care about is Now, a certificate alone is not enough for authentication. When we

want to use our certificate, we also need to prove that we control the key pair that is within that certificate. So, we need to use our private key. This is what we call a proof of possession ceremony. In TLS, this is kind of hidden away from us. Now, our X.509 certificate also has a signature on it. This covers the content, so the identity attributes and the key

pair, the public part of the key and this allows us to validate that the certificate was issued by somebody that we trust. If we take a look at an X.509 SVID as humans, it's not particularly pretty, but we can decode it. And when we decode it, we're going to have to cut loads of it out because otherwise it won't fit on our slides. Um I think this

says something about how much is in X.509 certificates that's not particularly relevant to all use cases of X.509. Probably most interesting to us is the subject alternative name. This is where we actually store our in an X.509 SVID. But we'll also find interesting the the subject public key info. This is where the public key that belongs to the workload would be and the one that it's going

to use when it needs to prove that controls the key for this certificate. How are X.509 SVIDs used in the wild? Uh almost entirely with TLS. I've met one or two people doing things that aren't TLS with X.509 SVIDs, but those are fairly rare. When we're using a certificate with TLS, it can play a few different roles. Uh most commonly in kind of like web PKI, right?

It's acting as a host certificate. So, the client knows that it's reached the right server, it's not kind of been misdirected. However, in the workload identity world, we also see it used as a client certificate. So, replacing the use of tokens and other kinds of credentials, identifying the client to our server. Even more commonly, especially with service meshes and the like, we see it playing both roles,

so mutual TLS. Now, TLS is a really great technology. It kind of gives us a lot for free. Um one of the great things about the TLS handshake is obviously it's leveraging the magic of public key cryptography. We don't expose our private key to the other party when we want to authenticate. What we also get after this handshake is a secure channel that the rest of our

protocols can run within. So, HTTP or gRPC. And this secure channel gives us confidentiality, so someone who uh intercepts our connection can't see what's being said. And it also gives us integrity protection. So, if someone comes in and tries to change something that's being said, the parties are aware of that fact. Now, I would say the pros and cons of X.509 are more the pros and cons

of X.509 in partnership with TLS. Uh the biggest pro, right, is just these are extremely mature technologies. I think it's probably decades at this point, right? Um and I tried to do some back-of-the-napkin maths, but I figure it must be, you know, hundreds of thousands, if not millions, of connections per second are being secured by these technologies. And that gives us a certain reassurance that not only

is it secure, maybe, um but also that it performs well, right? It's a good fit. Now, the other thing I like to say about TLS is those benefits we get on the secure channel are often things that we would want in our environments anyway. That confidentiality, that integrity. And from a developer's perspective, we kind of get them for free. You know, I don't know a lot about

the inner workings of encryption. Um most people probably don't. Some people do. I look up to them. Um but from a developer's perspective, when you roll out TLS, you don't really have to understand that nitty-gritty. You just kind of get it. Obviously, from a resource perspective, it's not free. It uses CPU and memory, but that's life. Um, so the cons, this is by no means a complete

list. Um, the first one is a topic called TLS termination. So, when we want to be able to use our client certificate to authenticate to the server, we need one uninterrupted TLS connection between both parties. In modern infrastructure, we see the use of load balances and proxies, and sometimes these will terminate our TLS connections. Now, we can then pass that client certificate forward in a header, but

this kind of introduces its own security concerns and challenges. We found that this was actually one of the major blockers for adoption in the organizations that already made heavy use of TLS termination or load balances, like layer 7 load balances. The other problem we find with X.509 is that whilst it's fairly common for software to support it for host authentication, it's fairly rare for it to support

it for client authentication. And even where it does support client authentication, the authorization rules that you can write in that software aren't powerful enough to say, "Hey, I only want to grant this specific workload access to a resource." Finally, I know some people will disagree with me, but I find X.509 TLS ASN.1, they can be a little difficult to work with. I think this is true for

most mature standards. Over time, more and more extensions and options are added, and it becomes harder and harder for someone coming in and trying to get grips with these software to understand the current state of play. Probably about a year and a half, a year later after the introduction of the uh X.509 SPIFFE ID, came the introduction of the JWT SPIFFE ID to kind of try to

address some of those challenges. Now, this is based on the JSON Web Token, so formalized by the IETF. You can pronounce it JWT, you can pronounce it JWT. Um, I don't mind too much. This shares some similarities with our X.509 certificate. We have a bunch of uh identity attributes. In the JWT world, we call this claims, and it's a JSON object, and we have a signature from

our issuer that allows us to validate the validity or the the veracity of this um token. Again, if we look at it, it's not particularly pleasing for a human being. We've got roughly sort of three segments of base 64-encoded JSON. But we can decode that, and we can format that nicely. The first section, we have our header. This contains information that a validator would need to know

to be able to validate the rest of the token. So, alg or algorithm here refers to the algorithm used for the signature, and kid or key ID identifies which of the issuer's key pairs was used to sign this In the main body or the claims, we have the stuff that interests us again. So, for a JWT SPIFFE ID, we say that the SPIFFE ID must be encoded

into the sub or subject claim. There's another claim I always like to point out, and this is aud audience. This identifies the intended recipient of the JWT SPIFFE ID. So, only example.com should accept this, foobar.com should not. Now, the keen-eyed amongst you or those with some background knowledge on this may have noticed one key difference between our X.509 and our JWT. And that's that the JWT doesn't

have some part of a a key pair, the public part of a key pair, encoded within it that belongs to the workload. And this makes our JWT what we'd call it a bearer token. To use it, we the client just needs to present it to the server. It's typically in HTTP, right? That's going to be in the authorization header that goes on each request from the client

to the server. What I will say is it's very rare to see the JWT used for uh authenticating a server to a client. Um, we tend to see sort of mixed usage where we use a X.509 certificate and TLS to identify the server, but then the client may use a JWT SPIFFE ID to authenticate to the server. The pros and cons of JWT. So, the first one,

I'd say the bearer token nature makes it very, very simple. Um, if you're using HTTP or gRPC, there's some sort of headers to put it in. Um, if you're using some completely custom protocol, there's probably a way to send some data when you start the connection or on each request if your protocol has the concept of requests. This also means that it works with most infrastructure. So,

unlike our X.509 SPIFFE ID where we're worried about TLS termination, our JWT SPIFFE ID will usually just pass straight through. Another benefit we get is uh compatibility with OpenID Connect. Uh OIDC is another set of standards I kind of initially created for human authentication within the realms of OAuth 2. Um, but they have a a concept called an ID token, and this ID token is also a

JWT like our SPIFFE ID. And it turns out that with a little bit of tweaking to our SPIFFE ID issuer, our JWT SPIFFE ID also become ID tokens. This has become particularly useful in the last few years. Um, most major cloud platforms, Azure, GCP, AWS, now support the ability for workloads to authenticate to them using these ID tokens. And this sort of gets rid of those long-lived

service account credential files and like that we needed to use before to have, you know, a workload on one platform speak to a cloud API on another platform. Now, our biggest concern here is the bearer token nature. Um, the biggest benefit and the biggest con. Um, this has a really weak security model. Whether a bad actor is able to exfiltrate it from the workload that holds it

or able to intercept it in transit, or, you know, able to pick out of weird places like log aggregation tools, right? Somebody in your organization may have accidentally logged their HTTP headers, and they've exposed that token. Um, there's nothing really stopping a bad actor from taking that token and using it themselves until it expires. This means that we need to use other mitigations when we're using JWT

SPIFFE ID. So, typically, we use a very, very short TTL. Uh like a JWT SPIFFE ID will be kind of in the order of minutes compared to in the order of hours potentially for an X.509 certificate. And this can place extra uh load on our issuance infrastructure. It's much easier to sign something once an hour for a workload than it is to sign it every minute. So,

with all this in mind, we needed to introduce something new. around 2 years ago, we took a look at workload identity today. And while X.509 and mutual TLS has the nice security benefits that they have integrity protection and encryption on the channel, most of most of the people would just use JWTs. Particularly in Kubernetes, you have service account tokens. They have the same security problem. They are

bearer tokens. The moment they leave the machine, they can still be used. Anyone who has their possession can use them. And X.509 has this great security properties, but it's very difficult to hard with to work with. So, that's why we kind of tried to combine both and create the workload identity token. It is defined by CNCF. Um, SPIFFE profile set calls it the WIT ASWIT. And it's

a JOSE-based token to protect a single hop in a call chain with cryptographic binding to the caller's That's a mouthful, but yeah, we'll get to that. There is a QR code to the standard if you're interested. It got into um influenced by some prior art. So, let's drill into it. It's a JOSE-based token. Same as the JWT ASWIT, it uses it's a You have the typical characteristics,

header, body, signature. Um, there's common claims, issuer, audience, expiry, JTI. And then there's this new claim called CNF, confirmation claim, and that is similar to the X.509 certificate. Um, it binds the public key of the workload into the signature. We'll get to that in a bit. What's also important is the type. Normally, you would see JWT or JOSE there. And that's actually a really bad practice, and

we need to get away from it because there's also logout tokens. And there's very common attacks to just use any kind of token as access token, and they just get accepted. So, today there's not really a way to differentiate those. So, you'll start seeing these type this type field being more specific in time. Yeah, it's a single hop in a call chain. If you see this call

chain, you would have foo call bar, but in between, you would have in this audience an ingress controller, TLS-terminating ingress controller, or some load balancer that does TLS This uh WIT is particularly designed to be called to put hops that go beyond So, the even though you terminate TLS at the boundary of your cluster or your other environment, the workload identity token is still going to protect

you from And cryptographic binding to the caller's identity. If you would have foo foo would have a WIT, which includes the public key, and it would also have a private key corresponding to that public key. So, every time foo calls bar, it would present the WIT, including a proof of key possession. It is me who the WIT was issued to. It wasn't some stolen WIT that someone

observed through a TCP dump. It was me because here's my proof. I have access to the key. The key could potentially be hardware-backed, software-backed, or what's not. Yeah, so the pros and cons. It's a key-bound credential. It requires proof of key possession, and it's not a bearer token, which is great. It is a well-established JWT format. You don't have to deal with ASN.1 encoding. And it's designed

to work with metal boxes. These are like These were our goals, and we believe we've achieved those. And that's why we believe it's actually a modern credential that can be easily used that is designed to work in cloud-native and Kubernetes environments. The cons, it's still an upcoming standard. We are still finalizing it, and of course, we've seen little adoption because it we're still like yeah, we need

to finalize it. There's also limited SDK support, but yeah, the SPIFFE community and the general the MC community is working on getting that supported. And by the over time you will see the workload identity token to be supported in your SDKs, your cloud tools, whatsoever. Yeah, so let's take a look. I have here uh SPIRE running. As I said before, as we said before, SPIRE is like

one of the implementations of SPIFFE that has a server and agent architecture. I don't want to get into details. You can look up the docs. What's important is um that workloads, they get a Unix domain socket. This here in temp SPIRE agent public API And on this socket are a couple of APIs Um one of those is the SPIFFE workload API, part of the SPIFFE standard. But

you would also see here this Envoy SDS. Um this is interesting because if you deploy Istio today, you're actually using SPIFFE under the hood. So, Istio and Envoy under the hood use SPIFFE. And you it comes with batteries included. But for example, if you want to use Istio across clusters, >> [music] >> then SPIFFE is a potential solution for On the workload API, API. You have a

couple of methods. Um you see here there is uh short variant, workload identity token variant, and X.509 variant. So, these APIs can be called by workloads, most of the time SDKs, to either receive a short, a WIT, or an So, if we call one of those, X.509 for example, you would see here the DER DER encoded X.509 certificate along with the private key and the signing keys.

Public part, obviously. Same with short. This would be a short ASVID. And this is the new part is that there is now this workload identity token ASVID. This one. And here you would actually see a short and the corresponding private key to it. And if we take a look at look at the short token, you can see in here it's a WIT ASVID. It has the corresponding

public part to this private key here. And the normal SPIFFE subject ID. Yeah, so any workload that is SPIFFE enabled today can just use these new APIs to now receive a WIT instead of a short on X.509 and can use that more in cloud native environments. Yeah, so what's next? Workload identity tokens, they're on their way to becoming an RFC. We're hoping to see adoption prior to

that, also receive feedback, but we are finalize in this document or like the standard. WIT ASVIDs, they also then once this has become an RFC to become part of the stable SPIFFE standard. Currently, it's in preview. And along the way we'll also like have SDK support, more examples, we'll go to more conferences, et cetera. If you're interested in learning more, we have a project pavilion this afternoon,

or we also have a Slack community. one more call talk about SPIFFE. Unfortunately, you've missed the first two and you are at the third one. Um which is SPIFFE meets OAuth. Um this is mainly about how to use SPIFFE in combination with OAuth. Very interesting how to bridge the gap between machine identity and user This is us and yeah, thank you. >> [applause] >> Yeah, we have

around 2 minutes left. Um so, the question was whether an extra hop is necessary to use the WIT ASVID. Yeah, uh no, it's not. The workload gets its private key and it can use it as long as the WIT is valid. Most of the time that would be for example an hour, but particularly as this one is now key bound, you can actually expand that and say,

"Okay, because it's key bound, I make it valid for a day." Because normally you would have these very short-lived credentials when they are bearer credentials because the moment they get stolen, they can only be used in that time. But if you have a key bound credential, you may actually have them expire a bit later because they're key bound and even if the actual credential is stolen, the

attacker would not have the key. So, you actually may you may also use this security capability to increase the resiliency for example, if you would have a network outage and your signing keys would not be available. Um the question was for proof of possession if there's extra round trips required. So, in order to provide proof of possession, there's currently two options. One of them is HTTP message

signatures, which is also a standard to actually put the signature on the HTTP HTTP request. And that would not require any round trips. There's no challenge or something. You just basically put a signature over the request and send it along. The other one is a a another short token, which basically is a short token that gets generated on the client on the HTTP client that would basically

provide the proof that I have key possession. So, in both cases, there's no challenge response necessary. Um there may be a third proof of possession capability in the future which would have a challenge response. to totally um outside of the actual WIT format, how you actually then provide the key proof key possession. One last question. >> You still have a question? Ah, okay, cool. Because it's uh

if you would take them Okay, so the question was if the second approach and it that it would be a second short, um if that would then again turn into Somewhat yes, somewhat no. So, if you take if both of them are compromised, if you key proof of key possession and the WIT ASVID are compromised in a call, you may replay the call again and the audience

would still match the original caller. That is a problem that you would normally not have a challenge response because with every call you would have a new challenge. The thing there is that um the proof of key possession, you would have that bind to as much request context as possible, right? So, for example, if you would have a content digest, you would try to put that into

the signature as well. So, even if you modify the content a little bit, you would have a different content digest, signature would not be valid anymore. Yeah, it really depends on your security boundary how much or like your proof of key possession, how much part of the request is it going to cover or not. Okay, so we are time. If you have any questions, there's a project

pavilion later this afternoon or tomorrow, or just yeah, hit us up on LinkedIn or on the SPIFFE Slack. Thank you.