It's been 5 years since I wrote my first post comparing the number of CVEs (security vulnerabilities) reported each year by major programming languages and web frameworks and asking Who's Had More Vulns?

Who's Had More Vulns- PHP, Java, or ColdFusion?

And it's been 2 years since I went back and updated the numbers again to reflect the newest data, still showing there is no statistical support for the notion that CFML is somehow inherently "less secure" than other languages. 

Who's Had More Vulns Redux- PHP, Java, ColdFusion, ROR, or .NET?

The Rub

There's a fair amount of disingenuity, or perhaps just willful ignorance to the statistics here and I talk to a lot of people who are astonished that CFML is still in use due to the alleged massive numbers of vulnerabilities.  One would think simply touching the code might give you rabies.  Some CFers suggested that it's a form of "virtue signalling" by infosec professionals to throw CF under the bus, winking at each other over there shared distaste for a platform they have little knowledge of but assume sucks.

I was brought back to this blog "series" this week after coming across this tweet by someone claiming all use of CF should be dropped due to "the number of critical CVEs it generates for very little return" and not being a "real framework" (He didn't explain what that last part meant)

Who's had more vulns?

On that note, I decided to dust off my links to www.cvedetails.com and get the latest data regarding just how many CVEs are being reported by the major web technologies each year over the last 10 years.   Is CF really the worst there is?

Click to enlarge

The Facts

No, it isn't.  In fact, ColdFusion is decidedly average when it comes to the raw numbers of CVEs that come out every year.  Here you can see the last 10 years of CVE reports as publicly viewable on www.cvedetails.com showing the trend of yearly vuln reports for ColdFusion, PHP, Apache Tomcat, Java, ROR, .NET, Python, and Node.js.  Python and Node are new additions.  They have fewer vulns than CF, but I wanted to hit all the major server-side technologies from W3Techs to get a good data set.  Java and PHP tip the scales as usual with a huge amount of reported vulns every year.  This however, is never held against them, at least not in the same manner that people accuse CF. 

The rest of the pack is honestly pretty tight.  All the major web technologies have a few vulns every year, some more than others, but there are really no stand outs other than Java and PHP.  CF is comfortably "in the pack" so to speak.  Mind you, I'd love to see CF have zero vulns every year, but it's just silly to pretend it's outpacing others in this regard.  

CVE Ratings

Another thing that's bothered me a bit is the severity assigned to CVEs. Adobe's security team assigns the severity to each CVE and they appear to just have a chart where vulns of a certain type receive a certain severity rating regardless of how exploitable the vuln is.  While erroring on the side of caution may seem safe to them, I think it causes unnecessarily bad press.  For instance, the entire series of RMI Java serialization bugs are only exploitable on servers with their RMI port exposed, which is a very small number of installations.  A simple firewall rule to only allow HTTP traffic squashes external access to this exploit.  In my opinion, that should reduce the severity rating.  Something should be 10 out of 10 if, say, 75% or more of all installations are sitting ducks to the attack. 

Then there are CVE's like this one which I don't think even deserves to be called a vuln, let alone a "critical" 10 rating.  That was, at best, an enhancement to the long standing behavior of uploading files to help protect unscrupulous developers who were storing uploads right into their web root.  Yet, Adobe security classified this enhancement as a CVE with the highest rating.  I am very thankful for Adobe's quick attention and seriousness that they treat all security issues, no matter how small, but I feel like they go a little overboard sometimes and security flaws are sometimes the only marketing Adobe CF gets in the greater programming space!  

I'm not sure if Adobe is using it, but the CVSS is a standard for rating vulns that takes into account (among other things)

  • Access Vector (what sort of access to the target machine is required to exploit)
  • Attack Complexity (How difficult is it to exploit)
  • Exploitability (whether exploits have been publicly released)

https://en.wikipedia.org/wiki/Common_Vulnerability_Scoring_System

10 Year Totals

Ok, time for another graph.  

Click to enlarge

Just reaffirming where CF sits in the standings.  Nestled between .NET/Tomcat and ROR with an average of only 8 CVEs a year, compared to 64 a year for Java and 35 a year for PHP.  (Note 2019 data is incomplete)

Takeaways

Ok, so some takeaways

  • ColdFusion (CFML) is not inherently less secure than other web tech stacks
  • ColdFusion (CFML) does not have a significantly larger amount of reported vulns.  In fact, the opposite is true.
  • Adobe tends to have a lot of vulns marked 10/10 Critical, but I think that's a little skewed due to their "abundance of caution" approach.
  • It's interesting to note that a lot of the CVEs this last year were in 3rd party Java libs and required direct access to the ports exposed by CF that a normal firewall would block
  • Adobe did a great job in documenting and fixing their vulns quickly. This is what you expect from a well-supported software. 

Raw Data

And here's my raw data for the graphs above:

  Java PHP .NET Tomcat CF ROR Node.js Python
2009 44 22 10 8 5 4 0 0
2010 54 35 4 8 6 1 0 7
2011 57 37 6 14 14 12 0 2
2012 47 22 13 15 5 10 0 5
2013 131 13 17 4 13 18 0 2
2014 115 33 10 13 5 12 1 6
2015 80 28 20 3 4 2 1 1
2016 37 106 9 12 5 8 9 5
2017 69 44 4 22 6 4 19 3
2018 56 21 12 8 14 0 18 8
2019 17 25 10 4 9 0 2 9
Average 64 35 10 10 8 6 5 4
Total 707 386 115 111 86 71 50 48

Sources