Whew, Mac OS X is VERY SECURE
There has been a lot of recent chatter regarding security and Mac OS X. Ed Moyle of Security Curve has an opinion that I've been following that I feel is on point. I live in Wisconsin, so when I saw this, I was naturally embarrassed. To be fair, the rm-my-mac competition was referenced and that contains a considerable amount of bias as well.
Let's lay the groundwork for the sensationalized headline that Mr. Moyle references. New Apple Hacking Contest Proves OS X Is "Very Secure" is the article written by Axxel (ok, he has a good moniker, so I like him for that) that PROVES in 11 paragraphs that Mac OS X is "very secure".
I've said time and again on this site, the scientific method should be applied to all studies. Let's see if we can't use Axxel's short essay to cover what we believe to be the state of affairs for an experiment:
- Hypothesis: The Apple Mac OS X operating system is secure. Dave Schroeder, a senior systems engineer at the University of Wisconsin, launched his contest Monday.
- Experiment Statement: For his challenge, Schroeder connected a PowerPC Mac mini to the Internet. The machine ran Mac OS X 10.4.5 with the latest security updates. The Mac had two local accounts, and Schroeder left both SHH and HTTP open.
- Experiment Tests: The mini garnered attention and lots of traffic, said Schroeder, who logged 4,000 attempts. The machine weathered two DoS attacks, various Web exploit scripts, SSH dictionary attacks, and untold probes by scanning tools, he added.
- Analysis: There were no successful access attempts of any kind during the 38 hour duration of the test.
- Conclusion: OS X Is "Very Secure"
Since I'm quoting from the article for each of these areas, I'll try to provide a bit of a backgrounder. Apple's Mac OS X operating system may indeed be "very secure". But one does not boast for a status of "very secure" after one 38 hour test. There are a number of statistics concepts that can be applied here. I'll name none of them and instead mention a few qualitative ideas. 1) Does everyone in the world with "hacking" experience on Mac OS X that could "break OS X" speak English so that they could be made aware of this "test"?, 2) Is everyone who has the hacking skills for this "test" aware of the test? 3) If I'm a hacker, do I want to play on a public honeypot? 4) If I'm a hacker and do have a 0day exploit, is this the place to make that known? 5) Was every hacker that was capable of penetration for this test at their keyboard ready for this challenge with no other day job, no other commitments, not on vacation, etc.? 6) If I'm the hacker with knowledge of a vulnerability, are there ways to capitalize (think criminals and money) on my work in a more fulfilling way?
Good thing that there is no bias in this test. Data must support the conclusion. Could someone please step up and tell me how many exploit attempts per day occur on the internet? Is 4000 attempts over 38 hours a reasonable sample? Are each of the 4000 attempts independent observations?
I wonder how the crypto community would feel if a new cipher were introduced and certified as "very secure" on some web site after 38 hours of open competition (i.e. all of the best cryptographers in the world weren't directly invited to participate; assuming that they would want to particpate) to determine its strength.
Did everyone believe that Fermat's Last Theorem was "true" and just accepted it as true because no one had found a counter-example?
The work to move from the above analysis to conclusion is essentially "no counter-example was found". The conclusion is dangerous given the data. The pragmatic or practical perspective I'd state is:
- If it is in the wild, it is fair game. In other words, tests like this prove nothing. At the end of the day, vulnerabilities exist. To find them may be difficult, but to declare a platform "very secure" under the above conditions is irresponsible.
It is difficult to separate the zealots and the hype in these stories. Look past those distractions to sensible practices and common sense. Security Curve Weblog and other links on my blogroll are a good place to start for some day-to-day reading on security topics.
By the way, Mr. Moyle and Diana Kelley have a recent book, Cryptographic Libraries for Developers, that looks like a good piece of work. Given their opinions, this book is one of my must-reads and I'll be ordering it shortly.