I’ll be honest: It’s been quite a while since I seriously worried about anything in cybersecurity other than software vulnerabilities. Almost every serious cyberattack you can name in the last say five years, including Not Petya, SolarWinds, Kaseya, and literally every ransomware attack, was either based on or enabled by at least one software vulnerability.
Of course, when the average cybersecurity person thinks about software vulnerabilities, they probably think of badly-trained (or simply incompetent) software coders who make mistakes and leave vulnerabilities in the software they’re writing – or else malicious actors who plant backdoors (which are just deliberately-inserted vulnerabilities) in the code during development, so they can later come back when the software is deployed and exploit those backdoors. Certainly, both types of people exist.
This was certainly what I thought of when I thought of software vulnerabilities, until I fell in with a bunch of people – loosely, the “SBOM community” – who have been on the front lines of fighting software vulnerabilities for years. These people know that no amount of training, pleading, threats, firings, etc. will ever cause developers to stop creating new vulnerabilities. As the Good Book says, “Software vulnerabilities you have always with you.”
Given that there will always be new vulnerabilities, what’s most important are identifying vulnerabilities, patching them as quickly as possible, and making users aware of both the vulnerabilities and the patches. But there are some serious obstacles standing in the way of accomplishing these goals – and for once, the individual developers aren’t to blame for them. In fact, the most serious obstacles may be due to problems with the various institutions that we’ve created, in a well-intentioned effort to mitigate the risks posed by software vulnerabilities.
One of these institutions is the National Vulnerability Database, which was created in 2005 by NIST and continues to be run by them (MITRE Corp. created the CVE List in 1999, and still maintains it. That list is continually incorporated into the NVD, but isn’t in fact part of it). A foundation of the NVD is the CPE (Common Platform Enumeration) name, which is meant to be a unique machine-readable identifier of a particular version of a particular software product.
CPE is a crucial component (so to speak) of the software vulnerability management ecosystem. This is because the only way a software (or intelligent device) user can learn what CVEs (and by the way, CVE used to stand for something, but now it’s NBI – nothing but initials) apply to a product they use is to search the NVD, using the CPE name for the product (although they may first have to search for the CPE name itself, using a supplier or product name).
In theory, the supplier of your software product should provide, in their SBOMs, a CPE name for every component of the product. Armed with those CPE names, you (or more specifically, a tool that you operate or that a third party operates on your behalf) can search for any CVEs that apply to each component. You repeat this for each component, and you’ll end up with a list of all the CVEs that apply to at least one component in the product. What could be easier?
Well, it turns out that there are a lot of problems with this system, which prevent the user from getting accurate answers. I addressed a few of those problems in the only post I’ve ever written on this issue; these are collectively known as “the naming problem” in SBOM circles. In those circles, the naming problem is thought to be practically insoluble, although it can be managed – just like an incurable disease can often be managed, so that it won’t automatically be fatal.
Until last Friday, I agreed with the idea that the naming problem could be managed, so that the patient wouldn’t die of it. But on Friday I saw a great presentation that Tom Pace of NetRise made to an informal meeting I was part of. While there are a lot of problems with the current CPE/CVE system for identifying software vulnerabilities, Tom pointed out one that perhaps outweighs all the others – and makes me and a few others think that the naming problem needs to be solved in the near future, not just mitigated.
Before I explain the problem, you need to know that NetRise is a company dedicated to firmware security risk management. But just like software (certain theologians argue that firmware is software; however, I don’t get involved in theological arguments), probably the biggest source of risk in firmware is the components included in a firmware product. Also like software, in order to learn about vulnerabilities in those components, the user needs to have an SBOM listing them, with hopefully a CPE name for every component; then the user can look in the NVD to find the complete set of vulnerabilities (CVEs) found in those components.
Now, I already knew that there were problems with looking up CPEs. I mentioned a few of these in the post from 2020 that I referenced above. And since I anticipate I’ll do more posts on this problem as the effort to solve it moves forward, I expect to discuss others. However, the problem Tom discussed was one I hadn’t heard of before, although it follows so clearly from the facts (all of which I already knew), that I wonder why I didn’t think of this.
Let’s recap where we stand now: For those of you keeping score at home, the problem I’ve just described is that probably most software products in the US don’t appear in the NVD. This means that a user of one of those products, who wants to learn about vulnerabilities in it, will never find a single vulnerability – but they’ll get the same message that they would get if in fact the product was in the NVD (with its own CPE name), but it was truly vulnerability-free.
You may wonder, “Perhaps most of these products never had a vulnerability, or at least they’ve had such a small number that it really doesn’t matter that they’re not in the NVD. Could that be the case?"
Tom says NetRise has found a lot of products (again, mostly products that are used as components of other products) that seem to have a lot of vulnerabilities, yet there’s no CPE for any of them. He pointed (by name) to one particular CPE-less product – used in many ICS environments – whose manufacturer has never entered a single CVE for that product. Is this because they’re such careful coders that they never make mistakes? Not likely.
But it’s even better: That manufacturer is so good that there exist no CPEs for any of their products (and they have a bunch of them). Is this because they’ve never found any vulnerabilities, despite constantly searching for them in all of their products, all of the time? Again, not likely.
And I’ll add that this supplier is so good that their web site doesn’t contain a single mention about vulnerabilities or even product security in general. Of course, this means they really have their act together, right?
However, Tom, being a naturally skeptical guy, decided to do some scanning on one of that manufacturer’s products – just to see if perhaps there might have been one or two vulnerabilities that had escaped their attention. The good news? He didn’t find one or two vulnerabilities. The bad news? He found 1,237 – in that one product. And he could probably have found a lot more if he’d had the time.
Boys and girls, this is a big problem. Here we’re worrying about individual vulnerabilities - like the log4j vulnerabilities - that are found in thousands of products. Those are legitimate worries, but what about the tens or hundreds of thousands of products that have never even reported a vulnerability – since they’re not even in the NVD in the first place. In fact, there are suppliers who make lots of products, that have never reported a vulnerability on a single one of them. And it turns out that one randomly-chosen product, which has never reported a vulnerability, in fact has at least 1,237 vulnerabilities now.[i]
I think Tom’s presentation makes it very clear that software vulnerabilities are a big problem, and the vulnerabilities that we know about are just the tip of the iceberg. But what would you tell me to do if I told you that this was just the second-biggest problem that Tom pointed out?...Well, I’m not going to do that, but I will describe that problem in one of my next posts. Stay tuned.
This blog is part of a 3-part series, view Part 2 here.
References
[i] Of course, it’s likely that a lot of those vulnerabilities will turn out not to be exploitable, but Tom doesn’t have the resources to validate each one of them. He said he’d have to hire 15-20 people just to do that. But even if say only 5% of those 1,237 vulnerabilities are exploitable, that’s still over 60.
Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.