Linux Swiss Cheese? Not Likely…

Linuxblog Editors Note: This article was written as a direct response to the report published by Security Innovation in March 2005 featured at various news sites.

It’s studies like the one discussed in this article that really hit nerves. Not for the reason that most people get upset about them…I hope people realize that there will always be FUD flowing from the patent pending FUD machines at M$ headquaters. One can only hope that as people become more and more computer savvy, that things will change in this respect. In the meantime, comments will flourish on subjects like this. But I’m not upset about this subject for the reasons that people think. I’m not an open source zealot proclaiming the GPL from the pulpit; rather, I am a supporter of Open Source, a user of Open Source, and a believer in Open Source. The reason this article makes my teeth gnash is the level of professionalism in the report it is discussing. What? Two Noted Doctors in the computer science field and a security author lack professionalism? You’re 100% right on that one. If they were professional, they would have made unbiased decisions AND stepped down if a conflict of interest existed. For more info, read on:

Let’s do a quick rundown of what the article is trying to say. Basically, the article states that there is something called “days of risk” that signify how many days an enterprise server has vulnerabilities that are unpatched. The study found that Microsoft 2003 server was only vulnerable for average of 30 days while Red Hat ELS 3 was open for 71 days. Considering the way in which vulnerabilities are addressed at Microsoft and their approach to solving said vulnerabilities this is highly unlikely. Let’s not worry about these ‘days of risk’ to begin with. Instead, let’s take a look at some interesting tidbits of information that one can find by reading the report and cross comparing it with a couple of websites.

1. The researchers used Red Hat ESL 3 ES which is the bottom line version of Red Hat’s offerings…not the premium offering of RHEL 3 AS. Right away they’re comparing apples to oranges. If I am going to publish a report on a comparison of security of two software vendors I’d be 100% certain I had their #1 player in the field. In this case, the researchers didn’t.
2. The Linux comparison was made using a MySQL database, which is NOT what Red Hat Enterprise Linux uses for its database. This means that the researchers went with a third party database for Linux while going with MSSQL Server for Microsoft. This is clearly a point of bias. For those interested, Red Hat Database 2.1 is a modded version of Postgres SQL. While MySQL is probably installed by default (help me here readers) it is not the recommended and default database. Therefore, the researchers should not count any vulnerabilities for MySQL but rather, Red Hat database 2.1. By not doing so, the researchers have shown their incompetence with open source. They even admitted as much in another article on the same subject citing “that MySQL had five vulnerabilities that took more than 90 days to fix” which was directly counted against RHEL in the comparison. They also acknowledge this fact directly in their paper but for some reason they don’t seem to think it is a problem. Perhaps they had some Redmond in their eye.
3. Dr. Thompson and Fabien Casteran work for Security Innovation, which is a company that does a ton of work for the DoD. Guess what software and company they deal with most of the time? You guessed it. Microsoft is at the top of their list.
4. Dr. Richard Ford, a researcher for this report, is a research professor in the computer sciences department at the Florida Institute of Technology’s College of Engineering. Guess who funds the department at the Florida Institute of Technology? You guessed it again. Microsoft is at the top of their list.
5. Default scans with nmap inside the report show 6 TCP and 4 UDP ports open on default install for RHEL 3. The scans show 9 TCP and 7 UDP ports open for a default install of MS Server 2003 install including my favorite TCP port 129 ;). MySQL is one of the open ports for RHEL and all of the vulnerabilities of MySQL were included with it in the report. Not only that, but they don’t state which version of MySQL they install even though they identify Windows SQL Server 2000 as the version installed for Microsoft. Nice eh? Based off of these port scans, somehow Linux has more vulnerabilities than Microsoft despite having less ports open. Very odd indeed.
6. The report does not state what patches were applied, installed, or in place at the time of testing for either Red Hat Linux nor Microsoft.
7. The report does not state the fact that Red Hat EL 3 ES comes only with a general support package while the premium support package comes with AS. AS is guaranteed quicker patch applications and issue resolution.

As one can see from the information above…the researchers dropped the ball on this one. Sure they have fancy doctorates and degrees…sure they’ve been in the field for 20 years. All of that and a subway token will get them on the subway. It all comes down to the fact that they didn’t know what they were getting themselves into and they made mistakes…the first of which should have been a conflict of interest. It’s kind of like the drug companies do when they are developing a new drug…the wheels of the researcher are greased a bit more than normal to give glowing reviews for the developed article…even if this isn’t the case here, the conflict of interest still exists and being a professional, the researchers should have taken note of this and done something about it.

Can a researcher remain objective while having cash flow from Microsoft? In a perfect world where morals and ethics were golden, yes. In the real world the answer is a resounding NO. Had it been done by a university or think tank that has no “alliance” or “partnership” with Microsoft it might have been less criticized by the open source crowd and general populous. Insofar as most people are concerned, this “report” will hold water for them. For that, I am generally sorrowed.

Some, including the researchers, might be saying…”well, the facts just don’t lie.” What makes a fact a fact? Two things. Knowledge and Information based on real occurances. When you start with a faulty premise at the beginning, you doom yourself on both correct information and accurate knowledge. In this case, the researchers chose to install MySQL as opposed to the official database utilized by Red Hat EL 3. The researches chose to review RHEL 3 ES as opposed to AS. Many mistakes that create an uneven playing field. The bad part about the whole thing? Now whenever those names…Thompson, Ford, and Casteran…pop on a report or study, they will be remembered as researchers (who start with crumbling foundations) who base conclusions on faulty data. Well, bad for them anyways.

But what of their “Days of Risk”? They’ve utilized an obscure and unrecognized computation method to calculate exactly how many days a system goes before it is patched by officially recognized sources. The days are accumulated after the vulnerability is announced. How convienient is this for Microsoft? If anyone thinks that Microsoft is going to be excited to publish a security hole in their operating system, you’re vastly delusional. Directly contrast this to the open source world where vulnerabilities are welcomed. Why? To make a stronger and less vulnerable operating system. People are motivated to announce vulnerabilities in open source BECAUSE THEY CONTROL when the patch comes out. As for Microsoft, the vulnerabilities are kept under wraps as to not alarm users. A really great article that examines this concept of disclosure of vulnerability is published on attrition dot org. I recommend it even though it was written quite a while ago and while some of it doesn’t apply today in practice, the concept is still there and valid.

Red Hat’s own J. Cox responded to this report on an average 31 “risk days” for response: “For example out of the dataset examined by the report there were only 8 flaws in Red Hat Enterprise Linux 3 that would be classed as “critical” by either the Microsoft or Red Hat severity scales. Of those, three quarters were fixed within a day, and the average was 8 days. A critical vulnerability is one that could be exploited to allow remote compromise of a machine without interaction, for example by a worm.” I realize that the researchers may be using “the new math” here but how does one get from an average of 8 days to 31? One would expect them to be in the ballpark here but it doesn’t seem like they are even in the stadium parking lot let alone the ballpark.

So where does this leave us? We have a faulty mathematic formula that isn’t disclosed in the report that calculates ‘risk days’. We have an uneven playing field created by the unbiased (ha!) researchers. And we have a conflict of interest in the researchers as well. All of this amounts to researchers whose conclusion should have been that their report was full of holes…not the Linux server they tested.

“A great deal of intelligence can be invested in ignorance when the need for illusion is deep.”

Saul Bellow

Author: devnet

devnet has been a project manager for a Fortune 500 company, a Unix and Linux administrator, a Technical Writer, a System Analyst, and a Systems Engineer during his 20+ years working with Technology. View all posts by devnet