Re: forrester: Here is my theory

From: Martin Schulze (joey@infodrom.org)
Date: Fri Apr 02 2004 - 23:32:26 CEST


As I said, I'm not quite happy with the current response. Hence, I
took the time to read the Yahoo article in detail and wrote down my
comments.

In a Yahoo story a research document from Forrester was quoted, which
I'd like to criticise.

  0. http://story.news.yahoo.com/news?tmpl=story&cid=1738&e=2&u=/zd/20040330/tc_zd/123143

| In a [71]new report, Is Linux More Secure Than Windows? from [72]Forrester
| Research Inc., based in Cambridge, Mass., Computing Infrastructures Senior
| Analyst Laura Koetzle finds that both Windows and Linux ([73]news - [74]web
| sites) can be deployed securely. Microsoft Corp., however, fixes security
| problems the quickest--which is a good thing, since it also has the most major
| security holes.

Wrong: Major security holes in Microsoft products are more dangerous
       since the majority of users don't apply patches and so they
       stay unfixed, they are also more dangerous due to their
       supremacy in the operating system segment.

| Forrester found that many IT professionals believe that Linux is more secure
| than Windows, but Koetzle found that the real-world answer is more complicated
| than that simplistic analysis.

True: Emphasise that the real-world answer is more complicated than a
      simplistic analysis, of which Forrester has provided one as well.

| Koetzle believes, based on a survey of past security vulnerabilities, that
| security vulnerabilities follow a timeline--in other words, that they have a
| lifespan.

True: Well, one could say so, I guess. However, the life of a
      vulnerability only ends when all vulnerable machines are fixed.
      That however, is quite unlikely, and... well, an explanation
      doesn't really contribute to a good repsonse, I guess.

| In this lifetime, real vulnerabilities to attack are usually born with a public
| disclosure of the problem in a form like the [82]Bugtraq security mailing list.
| Next, the ISVs or open-source developers prioritize the vulnerability and build
| a stable fix for it.

Wrong: Real vulnerabilities to attack can be born with a public
       disclosure of the problem, but don't have to. In Black Hat
       groups, vulnerabilities are not disclosed to the public but
       exploited unshamedly.

Also: Open Source developers are eager to fix problems as soon as they
      can, with only very short delays between their disclosure and
      crafting a corrective patch. Even if the project developers
      don't respond as soon as other developers would like them to,
      due to the openness and license of the source code, another
      developer usually jumps in and develops a correction.

| Lagging behind these developers, unscrupulous hackers then start exploiting the
| vulnerability. However, it's only after one of them builds an automated script
| tool for unskilled vandals (aka script kiddies) that the number of attacks
| really takes off.

Wrong: Unscrupulous hackers start exploiting the vulnerability soon as
       they learn about it, which does not require its disclosure in a
       public forum. In fact, many vulnerabilities in closed-source
       products are exploited long before the real vulnerability
       behind a problem is disclosed.

| The real period of enterprise vulnerability is after these script-kiddy tools
| appear and before customers apply the patch. In other words, most real-world
| security breaches on either operating system could be fixed with timely patch
| management.

Wrong: The real period of enterprise vulnerability is after a security
       problem has been discovered, regardless of whether script-kiddy
       tools exist or not. Script-kiddy tools only simplify exploiting
       the vulnerability a lot. They don't start exploition of the
       security vulnerability.

Also: For Free Software, most vulnerabilities are fixed in time. The
      majority of users of Free Software usually takes care of their
      systems and keep an eye on security and install security updates
      from the vendor shortly after they are released.

Also: Fortunately, there are also less script-kiddy tools available to
      exploit security vulnerabilities in Free Software.

| But the fault doesn't lie entirely with sloppy system administration, according
| to Koetzle. "It's up to the customer to apply it (the patch)," she writes. "But
| doing so isn't a simple task: Because few firms stick to consistent platform
| configurations and most lack robust testing and deployment procedures, patch
| application can take months--or longer. For example, for the nine
| highest-profile Windows malicious code incidents as of March 2003, Microsoft's
| patches predated major outbreaks by an average of 305 days, yet most firms
| hadn't applied the patches."

Wrong: The article only talks about major outbreaks of worms and
       viruses that exploit one or more vulnerabilities automatically
       and re-distribute the exploit through the network, adding
       damage to a lot companies.

       The article fails to address the fact, that vulnerabilities are
       exploited by black hat people, formerly referenced as
       unscrupulous hackers, withouth automated tools.

Also: GNU/Linux distributors have developed easy to use tools to
      update ones system with regards to security problems. Major
      GNU/Linux distributors also don't introduce new code with
      security updates so that after installing a security update the
      operating system behaves the same as before, it just contains
      one (or more) fewer security vulnerabilities.

      The overly behaviour of the software which is corrected with a
      security update is not changed. Major GNU/Linux distributors
      have established this policy[1], knowing that administrators and
      users who depend on certain behaviour won't be able to install
      security updates if they knew that their system changes too
      much.

  1. http://www.debian.org/security/faq#oldversion

| Forrester believes, though, that the judging of how well operating system
| vendors deal with security problem is bigger than just quick patch release and
| how well the vendor enables administrators to apply those patches. To Forrester,
| the key questions in judging operating systems are: how quickly does an
| operating system vendor fix public security vulnerabilities; how severe are
| those problems, compared with other vendors; and how close the vendor gets to
| fixing 100 percent of its security flaws.

Also: For Debian, a survey from 2001[2,3] revealed that
      vulnerabilities detected and posted to the Bugtraq list and
      those sent as Debian Security Announcements[4] have taken the
      Debian security team an average of 35 days to fix them.

  2. http://www.debian.org/News/weekly/2001/34
  3. http://lists.debian.org/debian-security-0112/msg00257.html
  4. http://www.debian.org/security/

      However, over 50% of the vulnerabilities where fixed in a
      10-days time frame, and over 15% of them where fixed the same
      day the vulnerability was disclosed!

      The security teams of other major GNU/Linux vendors have done a
      similar job and real numbers will back this up.

Wrong: The article talks about vendors "fixing 100 percent of its
       security flaws". With only very little knowledge in computer
       science, one would know that it is very difficult to create
       bug-free software. Most of today's software is much too
       complex to avoid all bugs and security problems.

       By correcting known security flaws, developers contribute in
       the direction of a less-buggier version, but it is quite
       unlikely that they create a bug-free version.

| To get quantitative answers to these questions, Forrester used two metrics. The
| first is the number of days between when a problem is publicly disclosed and
| when the operating system vendor releases its fix. In Linux's case, a component
| maintainer--such as The Apache Software Foundation for the Apache Web
| server--can patch security holes, but then there may be a delay before the Linux
| distributor releases the component creator's patch. Forrester calls this period
| the "distribution days of risk."

Also: However, many so called component maintainers and vendors of
      major components of today's GNU/Linux systems maintain close
      relationships to the security teams of major GNU/Linux
      distributors so that corrections to vulnerability flaws are
      provided in a very close timeframe after the vulnerability
      became known.

      Hence, there's often only very little time between public
      disclosure of a security problem and security updates by the
      various distributors.

      Due to the openness of the source code security teams from all
      major GNU/Linux distributions are helping software authors when
      they learn about security problems in their software in order to
      provide timely corrections.

| The second metric is the United States' National Institutes for Standards and
| Technology's [83]ICAT project standard for high-severity vulnerabilities.
| According to ICAT, high-severity vulnerabilities can be used for exploits that
| enable any of the following: 1) a remote attacker to violate the security of a
| system (i.e., gain an account), 2) a local attacker to gain complete control of
| a system or 3) the Computer Emergency Response Team Coordination Center to issue
| an advisory.
|
| Using these metrics, Forrester looked at security-vulnerability data for the
| period between June 1, 2002 and May 31, 2003 for the operating systems Debian,
| Mandrake, Windows, Red Hat and SuSE.

I'd be interested in the data behind this.

| Microsoft came in with the lowest average "all days of risk" with an average of
| 25 days between disclosure and fix release. In addition, the company fixed all
| of its security holes. However, ICAT classified 67 percent of Microsoft's
| vulnerabilities as high-severity, placing Microsoft "dead last among the
| platform maintainers by this metric," the report noted.

Also: Having read a couple of times on the Bugtraq list that people
      have reported security problems to Microsoft but no correction
      was created by the software vendor, I have much doubts with the
      above assertion.

| By comparison, only 56 percent of Red Hat Inc.'s Linux distribution's
| vulnerabilities were qualified as high-severity. Red Hat fixed 99.6 percent--all
| but one--of the 229 applicable Linux vulnerabilities. Red Hat and The Debian
| Project--which is run by Software in the Public Interest Inc., a non-profit
| group that runs a number of similar projects--were the fastest of the Linux
| distributors, taking 57 days to fix these problems. Debian had the least number
| of distribution days of risk for the Linux vendors but only fixed 96.2 percent
| of the vulnerabilities.

Wrong: As a member of the security team of Debian, I cannot believe
       the above assertion. The abovely mentioned survey by Javier
       Fernández-Sanguino Peña provided totally different numbers.

| MandrakeSoft had a poor days-of-risk showing, but ICAT numbers showed only 60
| percent of its flaws to be high-severity. The company fixed 99 percent--all but
| two--of its 199 applicable vulnerabilities.

I cannot comment on this but having worked together with the security
team of Mandrake closely in the past, I also doubt the alleged outcome
of the Forrester research.

| SuSE Linux, now owned by Novell Inc., did better than MandrakeSoft in resolving
| problems in a timely manner, but ICAT considered 63 percent of SuSE's 176
| applicable vulnerabilities severe. Of those vulnerabilities, SuSE only fixed
| 97.7 percent.

I cannot comment on this but having worked together with the security
team of SUSE closely in the past, I also doubt the alleged outcome of
the Forrester research.

| Based on these results, Forrester didn't come out with a single recommendation.
| Instead, the analyst firm recommends that businesses that value quick patches
| look to Microsoft and Debian. At the same time, though, Forrester is concerned
| that Microsoft's new monthly security policy may delay important fixes.

Also: I would rather recommend everybody who values quick patches gets
      in touch with their operating system vendor to provide security
      updates once they (the business) learns about them and applies
      them as well.

Also: For several security problems in variants of the windows system,
      which are exploited by recent worms and viruses, an update is
      not available for all versions of the windows system still in
      use and generally supported.

| If your business has relatively unsophisticated administrators, Forrester
| recommends MandrakeSoft, Microsoft and SuSE, since all three of these companies
| "hang their hats on the ease with which relatively unskilled users and
| administrators can install, configure, and patch their platforms," according to
| the report. If your staff is a step above that, Forrester recommends Red Hat and
| Microsoft.

Also: I guess that because Microsoft "hang their hats on the ease with
      which relatively unskilled users and administrators can install,
      configure, and patch their platforms," there is this
      comprehensive list[6] of uncorrected vulnerabilities in the
      Microsoft Internet Explorer?

   6. http://www.safecenter.net/UMBRELLAWEBV4/ie_unpatched/

      I guess this is also the reason why the German Heise Verlag has
      implemented these browser checks[7,8,9] and asserts that
      Microsoft still does not provide a patch against this
      vulnerability?

   7. http://www.heise.de/security/dienste/browsercheck/demos/ie/e5_19.shtml
   8. http://www.heise.de/security/dienste/browsercheck/demos/ie/e5_20.shtml
   9. http://www.heise.de/security/dienste/browsercheck/demos/ie/url-spoof.shtml

Thanks to youam and Tolimar for the references of out-standing
corrections by Microsoft.

Regards,

        Joey

-- 
Life is a lot easier when you have someone to share it with.  -- Sean Perry


This archive was generated by hypermail 2.1.7 : Fri Apr 02 2004 - 23:36:22 CEST