said by SpannerITWks :

. . . You will see how being able to write RK's and dissasemble them etc, and write detectors enables both sides to have a greater understanding of each others tactics etc. Thereby enabling them to design better RK's + detectors.

So i do believe it's definately worthwhile to as much inside knowledge as possible about how the other side Really works, because that IS what they do, every day !

said by Sunbelt's blog :

Publications need to use industry-standardized methods for testing. Organizations like Virus Bulletin have been doing this for years. Why can';t publications follow their lead?

said by Steve :

Consumer Reports has a different constituency than does the A/V industry, and it's not out of the question - in principle - to for one to wonder if if the A/V industry's tests are there to serve the industry and not the consumer.

Steve

said by Mele20 :

Is that from the article itself? It doesn't sound like a quote with the bad grammar and the repeated words).

said by Steve :

. . Consumer Reports has a different constituency than does the A/V industry, and it's not out of the question - in principle - to for one to wonder if if the A/V industry's tests are there to serve the industry and not the consumer.

Steve

said by eburger68 :

have all offered sound reasons -- methodological, practical, and ethical -- for not relying on lab-created viruses for anti-virus testing. The principles here have been rather settled in the AV community for some time.

said by funchords :

What has been conventional wisdom in past years may not necessarily apply to strategies for combatting today's 0-day malware.

said by HMS1 :

I don't see any problem with CR having created a bunch of new viruses. Obviously the bad guys can do the same whenever they want (the Bagle guys were cranking out the variations for a while, for example). CR's action did not increase this ability of malicious actors, nor would refraining have restrained the malicious actors.

said by HMS1 :

Retrospective testing is fine for anti-virus. But if you're working on heuristic detection, you may have good reason to write some new malware to test against. There may be some preexisting program that fits the description of what you want, or there may not be, or it may be too much trouble to find one.

said by eburger68 :

. . . the issue lies in creating new viruses that could, despite the best efforts of their creators, escape into the wild, and this kind of thing is a regular occurrence in the virus-writing world. The bottom line is, you don't compound the problem by writing these things yourself, even with the best of intentions, because intentions will matter very little if the thing escapes from the lab.

Eric L. Howes

said by eburger68 :

. . People being what they are -- which is to say fallible, gullible, ignorant, lazy, and prone to error --

. . it can be expected that the introduction of new viruses into the environment does increase the risk of people executing those viruses, if even accidentally. (sic)
And, by the way, in making this argument I am most certainly not slighting efforts to reduce the opportunities or chances for users to run untrusted code. We can do both: keep less open gasoline lying around AND keep people away from the gasoline.

Eric L. Howes

said by AB :

And it is, btw, 'common sense' (as well as mathematics) that says I am much more likely to be exposed to a virus if there are are 105,500 of them out there than I would be if there were only a couple of dozen!

said by HMS1 :

The danger increases with the number of viruses only if your behavior is careless. Instead focus on the possible ways they can get into the computer and be executed with privileges. The number of ways this can happen is far smaller than the number of viruses, and the list of infection routes remains far more consistent than the spectrum of viruses, and it is much more in your control.

said by eburger68 :

This is just a variant of the "guns don't kill people, people kill people" argument.

said by Steve :

Just because it's easy to accidently infect the world with ebola doesn't mean that nobody has the ability to create a virus test environment with proper precautions against leakage: I'm quite sure that both you and I would be able to construct such an environment.

said by Steve :

I am not arguing for the value of synthetic viruses like this (I don't know, I'm not an expert), or that CR actually did so in a safe manner, but it's not out of the question that they were aware of this issue and retained the proper experts to make sure that this didn't happen.

said by Steve :

It's perfectly fair to object to the testing methodologies on their merits, and to urge others not to play with fire, but this discussion about the "ethics" of this kind of testing smells incredibly self-serving: "We're the only ones who know how to do that"

said by eburger68 :

And when independent bodies demanded copies of those viruses in order to validate the test bed, which is a basic requirement of scientifically valid testing, what then? If you provide them then you have effectively become a virus distributor. And can you vouch for the safety of the parties to whom you distributed the samples?

If you refuse, then your test's results are invalid.

But the point here is that even the recognized experts in the field strongly advise against this type of behavior.

And as has been pointed out now, the issue of lab security goes beyond one's own security precautions, but the precautions of those to whom one might be obligated to share these synthethic viruses in order to establish the scientific credibility of the testing.

And I don't expect you or the other established security professionals would take kindly to having their own views characterized and dismissed in this manner.

said by Steve :

I am unmoved by this argument: I think it's possible to distinguish between a known virus researcher and Ivan J. Trojanovic - that kind of distinction goes on all the time by those with common sense.

said by Steve :

Data point: DSLR has a "Malware Archive" forum where people post samples of badware, but only those on a trusted list can download them for research. I'm on that list, you probably are too. Is Justin "distributing" malware? Or just using his head in an effort at public service?

said by Steve :

Sure, but advice that applies generally does not always apply specifically in every possible case.

said by Steve :

Example: The recognized experts also strongly urge people to run antivirus software on their desktops, but I never have. I'm very well educated in virus infection vectors, am extraordinarily careful, and have never had an infection in almost 30 years of using a computer. I'm not the only in this forum who believes this.

said by Steve :

The validity of the CR testing will stand or fall on the merits of the methodology, not the ethics. I'm so confident that the smart people like you will point out why they have reached an unwarranted conclusion that I simply do not care about the "ethics" issue. You'll demolish them without it, so to me it's just a distraction.

said by eburger68 :

And I, too, would expect CR to be able to make the distinction. But the point is, once they redistribute, those viruses are effectively beyond their control.

Bad comparison. Justin didn't create any malware and isn't re-distributing any malware that wasn't already in the wild -- that's the key difference.

1) Here the experts are urging the advice on each other -- the very folks one would expect to be the exceptions to the rule.

And the fact that the creation of 5500 new viruses lacks any practical or methodological justification makes the ethical lapse even more glaring. If they could mount a credible defense that such steps were necssary to allow some unique and innovative testing to proceed -- testing that might shed real light on the capaibilities of the tested AV apps and/or the behavior of malware -- then we might be looking at one of those ethical "corner cases."

But we're not. We're just looking at some run-of-the-mill irresponsible behavior by an otherwise respected testing entity that should have known better -- and all in the name of some rather unimpressive testing that shed light on little of anything except the organization's own ignorance and carelessness.

said by eburger68 :

For the sake of clarity, let's narrow the focus of our disagreement to this one statement which, so far as I can tell, motivates and undergirds your entire argument:

quote:

---

It just smells self serving to me.

---

How is the demand that entities that presume to do quality, responsible AV testing follow a simple ethical rule that one shalt not create malware onesself "self serving"?

said by eburger68 :

And when independent bodies ...

said by eburger68 :

...demanded copies of those viruses in order to validate the test bed, which is a basic requirement of scientifically valid testing, what then? If you provide them then you have effectively become a virus distributor. And can you vouch for the safety of the parties to whom you distributed the samples?

If you refuse, then your test's results are invalid. End of story.

said by eburger68 :

To return to the original issue -- the full issue -- which is whether CR had any justification -- be it practical, methodological, or ethical -- to create 5500 new viruses for testing

said by eburger68 :
in order to establish and defend the methodological integrity of the testing, CR would unavoidably risk compounding its ethical lapses

said by SpannerITWks :

What's the difference between someone sending a vendor X amount of New REAL nasties that they have discovered, that NO vendor has, and the vendors testing their AntiNasty with them, and then releasing Defs for them, and testing with those Specially written ones ?

Isn't that Exactly what would happen if those Specially written nasties got out somehow ?

Either a nasty IS a Nasty capable of doing whatever it can, or it isn't, then it ain't a nasty is it !

I believe those Specially written nasties are as valid as any others, that are new and discovered for the 1st time. Otherwise let's all pretend that ANY new nasties are completely irrelevent, and therefore we don't need protecting from it/them. Don't think so somehow !

said by GeekNJ :

And on a bit of a related thought, I've personally always felt that the A/V industry itself was possibly responsible for the scare and even creation of viruses in order to pump up the "We protect you from xxx,000 nasties". Of course, how many of those nasties have never been "in the wild"?

said by GeekNJ :

And on a bit of a related thought, I've personally always felt that the A/V industry itself was possibly responsible for the scare and even creation of viruses in order to pump up the "We protect you from xxx,000 nasties". Of course, how many of those nasties have never been "in the wild"?

said by eburger68 :

First, give part of Joe Wells' paper here a read:

"Lies, Damn Lies, and Marketing Perfidious Priorities"
»vx.netlux.org/lib/ajw01.html#p3

Joe rehearses one these cycles from the early years of the AV industry, when it was sucked into a competitive arms race over the number of "all known viruses" and the comparative detection rates of AV products. Bad research and analysis feeds opportunistic, competitive marketing, which feeds user fears and customer demands, which in turn feeds product testing and research, which feeds...

said by eburger68 :

The concerns that I've expressed -- that, in a worst case scenario, the industry and its customers could be drawn into enervating cycle lab virus creation and virus definition building in response to customer demands -- is not an idle one. We've seen these kinds of cycles before.

said by Steve :

But since Consumer Reports is not in the A/V industry, they don't have any of those incentives (they're not selling an A/V product), so these reasonable proscriptions on industry behavior do not apply here.

said by eburger68 :

Even if the AV vendors themselves somehow managed to refrain, we could still well wind up in a situation where AV companies were forced to contemplate whether to start adding lab viruses to their defintions. And who would benefit? Surely not ordindary users and consumers.

said by Steve :

It may well be that CR committed the crime in question, but the A/V industry is doing a terrible job in the witness stand.

said by SpannerITWks :

. . Anybody can test whatever they like, ther's no law against it, and as long as the tests are competent and the results are fair, and it's beneficial in some way/s to the users, so what !

. . just because we havn't been given the title " experts " or call ourselves that ?

Spanner

said by SpannerITWks :

How about Trojans + Rootkits etc then, is that OK lol ?

said by SANS Newsletter and editorial :

--Consumer Reports Creates 5,500 Viruses For Tests
(16 August 2006)
Consumer Reports is under fire from the anti-virus community for sponsoring the creation of 5,500 new viruses to test anti-virus products. Zone Alarm Internet Security Suite scored high in the test for both virus and spyware. Spybot Search and Destroy scored well for spyware.

»www.computerworld.com/action/art···_topic17

»cbs4boston.com/consumer/local_st···410.html

Special Tip: A great discussion on Microsoft Office security and vulnerabilities has been posted on SecurityFocus:
»www.securityfocus.com/infocus/1874

[Editor's Note (Paller): This controversy is especially problematic for the leading AV companies because they have traditionally not done well in finding and blocking new viruses quickly. But for goodness sakes, if they don't do well at finding and blocking new viruses, why ae we buying them? They should stop complaining and instead thank Jeff Fox and the editors at Consumer Reports for helping to do important product improvement research for them.