Unlocking product security certifications: Transparency through open source with sec-certs

This talk discusses important security certifications, specifically focusing on Common Criteria, EUCC, and FIPS 140-2/140-3 standards. The speaker explores the processes and challenges associated with these certifications, including the lack of transparency and the lengthy duration required for obtaining them. They introduce a research project from Masaryk University aimed at analyzing vulnerabilities across certified products and the development of a tool called Secterds. This tool facilitates the processing of Common Criteria and FIPS certificates and enables users to track dependencies among certified products. The speaker highlights the critical nature of these certifications in securing devices and the implications for security practices in the industry.

So, the first thing to question anyone because we are somewhere on the edge of research, we are research project, we do also compliance and Mercedes OC compliance today. So, anyone has any idea what security certifications are? Okay, that's great. Amazing. I didn't expect you know that many people you know to be So, yeah. What we are going to talk about today, there are you know two major

or now kind of like a free Interesting how to see myself from from the speaker. Uh so, we will talk about common criteria. it's an international standard, the ISO and I will show you how the certificate looks like later. We also now have the EUCC, that's kind of like an implementation of common criteria framework in the EU. You probably heard a lot you know about you know

CRA. Uh during this conference, so EUCC could actually be used as one of these you know evidence for for CRA for the critical products. So, that's quite important thing these days. And then we are going to show you a little bit about you know FIPS 140-2 and 140-3. This is initially North American standard, now it's international standard also under you know ISO standardization. So, I will you

know quickly you know introduce you to what these you know certifications are, how these certifications work. So, you will see you know why we are doing this project uh Waseem will later you know tell you more about the actual implementation. common criteria EUCC. You have so-called you know target of evaluation, that's basically that you know part of your product that's going to be evaluated later in the

process. Uh there are a few things, the security functional requirements, that's basically you know defines you know, your product implements from, you know, security standpoint. And then, you know, it's evaluated to the evaluation of the target of evaluation is the reality check between, you know, what's in the security target. That's the document that contains its, you know, SFRs and what's in your product. And then, we have

something that's called, you know, protection profile. So, the protection profile is more like catalog library of this, you know, SFRs. It's a template used. I will, maybe, you know, go now to FIPS. So, for FIPS 140-2 140-3, uh one note 140-2 expires this September. So, now it's only, you know, -3. It's the cryptographic modules validation program. Uh it's uh contains, you know, algorithm validation, module validation, and

entropy validation. So, we receive, you know, three uh certificates for different things. Uh very slight this, you know, limited set of approved algorithms and parameters like MD5 and other stuff. And this program is governed by CAVP and CMVP. people, you know, connect with the NIST, the US standardization organizations. We have various validation scenarios. So, I will, you know, quickly show you how these certificates looks like. Yeah.

So, now I can't see. Hm. Okay. I wanted to show you quickly, you know, how the certificate looks like. >> Kill the presentation completely. >> Uh kill it for me. Okay. So, this is, for example, one of the certificates. This is actually UCC certificate. There are, you know, these you know, award ceremonies. You can, you know, go and get this certificate on paper, but we usually don't

do it. So, this is how how the CC certificate looks like. And where is it? I made a mistake. Yeah. And uh This is, you know, how the FIPS 140-3 certificate looks like for the Red Hat Enterprise Linux 9 OpenSSL FIPS provider. You have all the information on this page. I can, you know, do it now. Yeah. I will just go back go back to the presentation.

That's the easiest way. So, you see this and now the the security target, one of these, you know, documents that contains everything I told you about. You see, it's a very, very long uh document. So, I will go back to the presentation. Uh demo time. So, don't do demos. Okay. And uh now there are, you know, some, you know, challenges in security certifications. So, one of the

biggest issues is lack of transparency. These things are non-transparent. These are PDFs somewhere, you know, on internet. You need to find, you need, you know, look up for these, you know, Super expensive things, especially for smaller vendors. It's something that, you know, if you want to achieve it, it's expensive from, you know, your time, resources, and money. And also, it's extremely slow. Like, the current queue for

FIPS certificates, and we are talking about security certifications, is up to 2 years. So, basically, you have like 2 years old OpenSSL library that's certified. And then, you know, this one with the stamp is the secure one. No, it's definitely not secure one. So, we are have this, you know, project, you know, to add more transparency, to make certifications cheaper and faster in the end. So, what

should >> Okay, and let me tell you now part of the story why we researchers in Masaryk University decided to care about travel going through thousands of certification security certification documents. We discovered one particular vulnerability that was about 9 years back that impacted passports in several countries, IDs in several countries like Slovakia, Estonia. Few countries did not launch or had to cancel their e-health card schemes like

Spain and Austria, if I do remember. And nearly 2 billion devices have been impacted. Very likely, your TPMs, trusted platform modules in your computers, as well as many, many others. And we were wondering then about one question that we originally believed was a very simple question. And as you know, simple questions usually have complicated answers. And that question was, "Okay, we discovered this one particular problem with

cryptographic library in one particular chip. How many devices actually get impacted that were certified under the Common Criteria standard?" To give you an idea how many products we are talking about, Common Criteria is a scheme that is in the existence for over quarter of a century. Over quarter of a century and about 6,600 products got certified. Uh Uh, when the scheme started, the certificates basically were lasting

forever. And then finally somebody decided that typically the products do not keep their security status forever. Vulnerabilities get discovered. So, the compromise was made certified product unless some particular problem is found will be invalidated for 5 So, you see two columns in this table and one carries the number of active certificates, one carries the number of archived certificates. once a certificate becomes archived, you still actually have

to pay attention to it and you will shortly understand why. And note that for some categories of products, you have dozens of them, for some you have just single units and for others, you basically take one quarter or well over quarter nearly a third of the ecosystem. And that's in particular smart cards here and integrated circuits. For these, you have like 20 to 100 of these products

being certified. These are the beasts you carry in your banking cards, some of them in your phones and the providers of your banking cards demand demand the security evaluation under the common criteria scheme. So, we are talking of thousands of certified products. Each of these carrying certificate as you have seen, many of them carrying additional certification documents. On top of that, the originally North American scheme FIPS

142 or 3 carries well over 5,000 certified products, cryptographic modules. About 1,000 of these certificates are still valid, uh, non-archived. So, we are talking all together of about 12,000 certified products and about 30, maybe maybe 25,000 certification Certification documents get published and get issued by individual vendors, countries, and vendors and countries have different cultures of producing Uh, for many producers or for some producers of even the

complicated function control C and control V seems to be beyond the level of automation that was achieved and not even interlinking of index numbers between the certificates is carried correctly. So, if you get a product that is certified, that's our Yellow Beast here, was certified under this certificate number by the German BSI, and then you know that if you discover a particular vulnerability, it impacts this particular

product. Then, if you go and if you know where to look, you will discover that this particular, uh, chip, microcontroller is actually used in at least three other products that got certified under the French scheme by ANSSI. And this way you can carry forward because one product often uses another, uh, for building up the security solution. And that leads us to investigation of references. So, before I

come to the definition of references and what they allow us to discover, let me provide you with a nutshell what effectively we do. We suck data. We suck data from the Common Criteria portal and we suck similar data from the, uh, of the US provenance. We put these data through some cleaning processes. Uh, to certain extent, we still use regular expressions. Now, we are transferring more towards

the LLMs. And eventually, we get to a tool that we are talking about today here, and that's tool called Secterds. Secterds, uh, is provided through several interfaces. You can use Secterds through the web interface and do your searches there. Uh, or if you are large corporation or national agency, you may not want the crazy researchers in Masaryk University to know what queries you are filing through Secterds.

So, you can install it on premise, and you can run it on premise. Secterds is provided as open source, obviously. And you can link our data that we suck from this portal together with your proprietary database if you are sharing one with your, uh, with your, uh, compatriots in some undertakings. We link it also together with vulnerability databases, so that a discovery for one's particular infrastructure, what

products you should worry about if vulnerabilities against these get discovered, is made much easier using Secterds. So, we are building, effectively, uh, as one of the important activities within Secterds, we are building, uh, a reference graph. Imagine this simplified certification documents that says that you certify certain smart card. This certification is based on a recertification of an older smart card and using its specific evaluation results. And

it also is using a particular, uh, integrated circuit that was certified, uh, in another certification process. So, you have two kinds of dependencies, predecessor dependency or component reuse dependency. Uh you should not worry about these dependencies anymore. Let me try to impress you with this particular graph that will come in a minute. And the graph displays the dependencies between the thousands of certified products. Each device, each

beast certified under common criteria is a vertex for us. Some of these will be isolated. That product does not depend on any other products in terms of the different dependencies defined before. Others, as you will see in a moment, create clusters. Our graph is of a reasonable size, 6,000 vertices and 3,000 edges. You get something like this. For those of you who sit closer, you can even

see small icons. These icons are displaying the type of the products according to the table that you have seen few slides back, whether it's a smart card, operating system, biometric authentication device, or something different. The products that you see here are not interdependent and do not create clusters, but you can see some clusters in certain locations. Where the icon is green, it's an active certificate. Where the

icon is yellow, it's an archived certificate. And as you can see, for example, here, you often have clusters of dependencies between active and archived certificates. So, what do we do using this graph? We do a lot of or we have a lot of fun with with this stuff. What we discovered, three top things that we be or we believe that are the top things that we discovered.

10 products, 10 out of the 6,600 are impacting 16% smart cards. If you recall from the table, smart cards create about a third of the ecosystem. I made the I made this calculation easier for you. We are talking of 220 plus smart cards are dependent on top 10 integrated circuits or microcontrollers. If any of them fails, a vulnerability gets discovered, you can count on dozens of products

with the low Sorry, with the high uh security certification uh by Common Criteria getting impacted. When we did this work for our Ruca vulnerability, we discovered over 50 other products like national ID cards, smart cards, and others being vulnerable to this particular flaw in cryptographic key generation. Uh one positive finding, higher reach, that is higher level of clustering, more dependencies on a particular certified product, is positively

correlated with this level of certification being higher, being done with a higher level of diligence. So, uh this is one of the positive findings that we made in the process. >> Okay, so in the nutshell, we have developed a pipeline for automated uh processing of Common Criteria and FIPS certificates uh plus, you know, the vulnerability databases, what you said. We do the dependencies between the certificates. We

have the continuous insights into certification ecosystem. So, if you know, like, you know, to follow, you know, closely what's going you can do it through Secterts. And of course you could see that we added more transparency into the ecosystem because previously it was basically impossible you know to do this you know search for affected products through this you know certification dependencies. The the biggest you know problem

in this area is like these you know documents were written by humans for humans. So the parsing extracting the information it takes you know some time it's like based and with LLM's but yeah. You will now have the API access to everything and you don't have to anything or you just can you know install it or use the web interface. Okay so what is the use what

are the use cases? If you are user Okay. So if you are user you can you know for example you know see you know what security claims are made for a product you can you know monitor certificates you are interested in let's say if you are waiting for some certificate to appear on the list I told you it can take you know several years you know to

see the certificates and you can't you know do watching the certifications pages everyday. One interesting part of the project is there is like heuristics to map CVEs to certified products. That was actually one of the papers if the certified products are more secure than non-certified products I you know recommend you to read them. And other analysis. Then for vendors let's say like me what's important I want

you know to see what our competition is doing. I can't do like morning tea and go to the you know common criteria portal FIPS portal and see if our competition is doing something suspicious we should probably you know follow too. So now I have a way you know how to get some business data out of you know the security project that it did in security team. And

uh for example, one example would be let's say if there is a new platform architecture, I don't know, AR64, the arm platform, I will see that my competition is doing something that I'm don't not doing and I will, you know, can follow them. So, uh for certification bodies, they sometimes, you know, don't like us because we bring more transparency and they believe it's not a good idea.

We believe it's a good idea. So, but they can, you know, let's say watch the performance of the ecosystem, what's going on. They can see if it's something suspicious coming from other, you know, certifications bodies because there is some, you know, kind of mutual recognition between national common criteria schemes and so on. And uh Certification laboratories. So, you probably as you know, most people, you know, know

what common criteria and FIPS Uh they are so-called certifications labs, the companies that provides the evaluation activities. They can, for example, see, you know, what other labs are doing, uh what are the trends, if there is a new protection profile, that means, "Okay, we probably need more knowledge because uh I don't know, smart cards are, you know, getting more traction." Actually, that's bad example now. So, yeah.

One of the things then for the government agencies, you as Wojtek said, you can install on prem and then you can enrich the data with, you know, whatever you need and then use it as like one tool for everything. Okay. And general public. Yes. Go to sectcerts.org, see, you know, what we can do. And uh yeah. So, one of the case studies, so I will now change

my hat from Red Hat to the Open SSL hat because I'm also member of the OpenSSL Business Advisory Committee. And that's you know, open source project. And we can you know, see you know, can we know get some you know, business data for the OpenSSL project? Like is OpenSSL still prevalent? How much well vendors you know, may use OpenSSL because you know, there's the OpenSSL Corporation that

basically sells services like subscription to OpenSSL and for them it could be you know, interesting question like if there there are there any you know, vendors we can come to and offer them our services. So that's one of the use cases and what are the committed competitors? So yes, OpenSSL is still pretty much you know, dominant in this area. It's like almost 17% of common criteria, 16%

of FIPS 140. And you can see you know, other libraries that very famous CVE Heartbleed that basically created a lot of you know, forks out of OpenSSL. So you can see that you know, some you know, of these you know, forks might be you know, getting more traction over time and it creates some you know, fragmentation in the ecosystem. So a very good data for OpenSSL, hey

do something about this or the whole community of crypto libraries. We don't need you know, 100 you know, different forks of crypto libraries, but we need one proper crypto library. So the data are very you know, clear that you can see the BoringSSL and uh other you know, libraries getting some So yeah, OpenSSL is dominant cryptography library in the ecosystem and you can use Sectors for these

you know, data insights. So it's not only you know, for security research, but also you can you know, get some interesting, you know, business insights uh these, you know, originally security And yes, one thing this is all, you know, information that's coming from over, you know, one 10K unstructured documents that are processed. Something that's impossible or impossible before. So, yes, use So, you can use Secterts to

go through common criteria certified products and associated certification documents. You can use it for FIPS 140, and you can use it for the new European EUCC scheme. Here you have a URL and QR code for the Secterts base itself. If you care about some of the uh findings we have done in the past years, in the right-hand side you have two scientific papers published in the Computers

and Security uh journal. And actually, uh Secterts was made possible using uh cybersecurity excellence hub between uh South Moravia and Estonia. Uh Secterts actually will be carried on through, and we will be uh we we will be testing it with uh devoted industrial partners, inclusive of Red Hat, in the forthcoming uh 3 years in the SECCAT project, where actually three other security tools open source tools will

be provided, and I will share a share a QR code for that project in a minute. And Secterts started in our undertaking we have done in the cybersecurity for Europe, where the original idea, "Let's try to do something with this mess of certification document," was originally conceived. Here you have the uh link to the current project SECCAT, where we are providing four technological targeting different levels of

the technology stack. One for FPGA security, one for TLS security, one for investigation of smart cards, trusted platform modules, and other security beasts beasts inclusive of cryptographic libraries, and one of them is the sector that we shared with you. So, many thanks for your attention and we'll be ready to answer questions. Thank you. Oh, and we also have got stickers. So, if you are interested in stickers

and you don't have enough on your laptops, come forward and grab some. >> Great. So, do you have question? Yes. Do we have questions? >> Hi. Thank you for your talk. It was pretty interesting. Just wondering the craft that you showed, how do you calculate the distances between the nodes? Because there were some on the right >> The distances technically are not important. Important is where the

products cluster, that is there is an edge between the two products. That is what is important, not the distances. >> Ah, okay. >> It's a matter of visualization and now we are trying different different techniques for displaying it because what you effectively want is to have the graph at your hands and be able to click and see how many other products depends depend on these beasts. >>

All right, thank you. >> Thank you. >> Do we have other questions? >> If not, why this is not an Eclipse >> Who knows? Maybe in the future. >> Okay. And another question is how much time it requires the computation of this? So, it's immediate. >> No, no, no, it's not not immediate, but it's I mean the searches and things are done on process data. We we

basically scrape the data a few times a week. We provide new new snapshots. We do the cleaning of the data so that the searches are significantly faster. Typical search from a typical connection will then take you single seconds. >> Okay. Okay, that's that's okay. So we are waiting for you in Eclipse Foundation and we can I think if you want to say >> to say we are

looking for contributors so we can work together. >> Oh yeah, if if any of the tools of seek it whether TLS scanner is one very particular thing that is of interest to many people I believe catches your attention, then reach out to me because within the seek it project we will be involving also non-commercial partners into the testing and you can influence what the tools will do

for the money provided for this applied research. So reach out to me if any of the tools is interesting. Thanks. >> Yes, this is a good information. Thank