Show me the virus metrics!

Rob Rosenberger, Vmyths co-founder
Monday, 27 September 1999 LET'S SAY VIRUSES got big in 1986, and let's say the web got big in 1996. Time for a simple comparison. Website metric utilities grew immensely popular in the last three years. Immensely popular. Corporate webmasters log every visit and generate all kinds of reports for their bosses. They can tell how many people visited on a given day, how many pages they viewed, how much e-commerce it generated, and so on. They pay big bucks for web traffic analysis tools with eye-popping charts & graphs. Webmasters archive their log files for posterity, too.

Virus fighters need to answer a simple question. "How do you justify your job?" These folks possess no empirical data and no virus metric utilities.

Compare this to virus metric utilities, which don't exist. (Technically, we shouldn't call this a "comparison.") Viruses got big ten years before the web came along, yet virus fighters can't generate a single chart for their bosses. Sure, antivirus software keeps an activity log, but most programs limit the file size by default. Old data gets overwritten just so it won't fill up your hard disk. Nobody really bothers to store this data for posterity. Why should they? No virus metric utilities exist. Webmasters can analyze web visits three years after the web got big. Virus fighters cannot analyze virus detections thirteen years after viruses got big. Doesn't this seem odd? Virus fighters sometimes fall prey to urban legends because little or no evidence exists to contradict those legends. You might actually hear a computer security expert spout "generally accepted facts" such as:

"The Morris Internet worm of 1988 cost $98 million to clean up"
"The Melissa virus crashed email networks at 300 of the Fortune 500 companies"
"The Chernobyl virus destroyed up to a million PCs throughout Asia"
"The ExploreZip virus alone cost $7.6 billion to clean up"
"Virus incidents in the first half of 1999 cost over $20 billion worldwide to clean up"

It gets worse. Do you know about the annual "ICSA Virus Prevalence Survey"? Computer security experts treat it as the Gospel. However, it relies entirely on input submitted by — you guessed it — virus fighters. How can these respondents support the claims they make?

If your "virus report" comes from a technician support product... it's probably not a virus report.

This problem stretches all the way to the White House if you can believe it. A 1997 presidential report declared viruses a serious threat without producing a single metric to back up the claim. I myself attacked the report for this reason. Still, President Clinton wants to spend more tax dollars on a threat he can't document.
VIRUS FIGHTERS NEED to answer a simple question. "How do you justify your job?" These folks possess no empirical data and no virus metric utilities, yet budgets and salaries continue to rise. So how do virus fighters justify their jobs? The answer to this simple question may stun you. First, CIOs don't expect computer security managers to produce virus metrics. They probably wouldn't even know what to do with such a report (at least not at first). Second, computer security personnel like to tell anecdotes. "Why, my folks removed a virus from the CFO's computer just last week..." Then they follow it up with a worst-case scenario: "you know full well what would happen if a virus exposed the CFO's bonus recommendations for next year..." Third, virus fighters cite national & international "surveys" like the ICSA Virus Prevalence Survey. (See above.) Fourth — when cornered — virus fighters can generate reports from technician support products, e.g. Remedy or Magic Help Desk. Those packages do contain some virus-related data, but they don't contain enough. For example, they don't count viruses detected on file servers or stopped at the email gateway. On top of this, they don't produce reports specifically geared toward virus metrics.

Don't let people sway you by claiming "the inaccuracies cancel out." How do they know erroneous virus help desk tickets cancel out erroneous non-virus help desk tickets? Two wrongs do not make a right.

Then, of course, we get into the same problems of unreliable data. Many "virus" support tickets have nothing at all to do with viruses, while many "non-virus" support tickets uncover viruses. Technicians notoriously fail to enter detailed descriptions when they handle a help desk ticket. (No offense to technicians! Their budgets & salaries seldom give them flexibility to keep PCs running smoothly.) If a "virus report" comes from a technician support product, it's probably not a real virus report.
THIS UTTER LACK of virus metric utilities will soon change. Trend Micro recently unveiled a product/concept known as "eDoctor," and VP Dan Schrader phoned me just to describe its report module. The skeptic in me put him through a meat grinder to make sure it will actually work — for example, it must first collect the data it will eventually analyze (no easy task). Schrader admits the eDoctor reports don't match up to a webmaster tool like WebTrends. Still, I believe it qualifies as an excellent start. I can't wait for other vendors to follow with their own report modules. God knows we need them. Unfortunately, we'll never truly know what happened in the last thirteen years of virus attacks. We lost the most valuable data of all — the beginning. Shameful. You know what upsets me the most? We didn't lose all of this data because of a virus...