Security → VirusP 's AV test - a simple question to u all

Since nobody was kind enough to email me (was it too hard??), to come and chat with all you members of this highly (?) respected forum, i decided to show up in here for another time and read your comments about my AV test. I haven't read all posts yet, and i know i will be seeing some certain people accusing me again and again, like they did 6 months ago and a year ago, of my poor judgement and criteria for making this test .. Some had even called me names in the past, names that i cannot write atm :P

Anyway, i just have a question for you admins and forum members, especially for those of you that KEEP ON ACCUSING me .. why don't you EVER have what it takes to invite me to join this forum whenever you decide to JUDGE ME, ACCUSE ME, OFFEND ME ??? Is that too hard for you people? I suppose blaming someone while he knows nothing about it is easy for anybody, yeah? I suppose i could start posting stuff about this forum and its admins (yeah, the official AV-related people that post in here and have very little credibility according to my opinion) in my own website's forum, without any of them to be able to make a response, even when they'd like to. But that's not my way as you can see. I am sorry you chose anonymity as a shield to accuse me. I am sorry for all those of you that keep on buying software that don't deserve your money and, most of all, I AM SORRY YOU BELIEVE THAT ALL OFFICIAL AV TESTS ARE 100% TRUE.

I will be waiting

Yours truly
VirusP

P.S. I was happy to see at least 1 of you has the guts to tell the truth about my tests. Thanks Stavros.

Stavros? I think you're referring to me.. My name is Jim..LOL..

No problem.. I did think of Emailing you, but I assumed someone already got to it.... I know, I know... I should NEVER Assume...LOL

Besides, I think you had others here rooting for you... among I would are Vampirefo and FF Again (The Thread Starter).

You did bring up a crucial point, however.. If in the future we get into another "discussion", we should make it a point to contact the person being discussed. Not everyone check DSL Reports 10 times a day LOL...

OK I was lost for a minute. Next time make a thread pointer so the rest can see the light of what your talking about. The one that went way OT (off topic).

»PC Utilities published VirusP 10-2003 av-test!

And I feel you are do the right to address this. But It will be up to the Forum Mod to let ride so folks keep it civil.

I'm sorry that thread is closed. Just wanted to say I have tried TDS-3 and it has detected the old Leaktest which was saved a year ago on my D:
No online virus scanner (and NAV2003 on the system) I tried has never found it.

Good to see you here Antony. Welcome back and I also hope this thread will be constructive. It seemed very difficult for many in that other thread to see what they were talking about since they did not have the magazine to read or the data that backs it up. But it was an intersting thread and many did try to provide facts and figures that came from the article as they were asked specific questions.

Difficult to do..but many times the questions can drive the answer as members desire to know how the product they use came out..and/or stacked up against the rest of the field in your test look at the most popular ones.

I did not see anyone stating they were going to switch product based upon the results.

Be Well,
John

zmaugy: If your speaking about Gibson's LeakTest, some AT developers put it in their sig defs as a demo "malware" since some of their customers got upset because their AT didn't detect it. BOClean did and I believe the Cleaner did also, if my recollection is correct. Don't specifically recall if TDS also did so, but it well may have since it detects it and it's not considered malware, just a demo.

said by sig6:

---

zmaugy: If your speaking about Gibson's LeakTest, some AT developers put it in their sig defs as a demo "malware" since some of their customers got upset because their AT didn't detect it. BOClean did and I believe the Cleaner did also, if my recollection is correct. Don't specifically recall if TDS also did so, but it well may have since it detects it and it's not considered malware, just a demo.

---

That's the one. Trojan Hunter has not detected it. I guess it can be explained like that (I thought about it as grc.com is quite known site), still it's purpose is to signal out of computer without user's request or permission.
Perhaps somebody from DiamondCS will tell us...

Okay, I don't quite get VirusP's point. Yes, he or she was trashed quite a bit in that other thread, and he or she certainly has the right to defend himself or herself, but what obligates anyone to notify VirusP that "hey, someone at Site X is discussing your tests" ??

If we have a thread that ends up bashing a Cringely column, or Bill Gates, or GW Bush, or Richard Stallman, is someone obligated to notify that person so that he or she may immediately join the fray at the DSLR thread? I don't get it.

The very fact that people were dismissing VirusP's work (rightly or wrongly; personally I have no idea since I haven't seen it) leads me to expect that they WOULDN'T bother to contact him or her. Apparently they have little respect for VirusP's efforts and don't feel it worth the time to "invite him or her to join".

Again, I make no judgment either way about VirusP and his or her tests, but his or her question here ("why don't you EVER have what it takes to invite me to join this forum") doesn't make sense to me. If you've published work to the world you have to expect that some may discuss it "behind your back", often without even thinking of soliciting your own input. The Internet's a big place.

-- B

said by B04:

---

Okay, I don't quite get VirusP's point. ........but what obligates anyone to notify VirusP that "hey, someone at Site X is discussing your tests" ??
-- B

---

I agree. I didn't understand the comment about "admins" either.

VirusP,

It's obvious that you put a lot of work into your test, however, I don't put any value in the results, as you probably read.

I would appreciate a simple answer to a couple of questions I have, if you are willing.

1. Were each of the virus samples used in your test executed on a computer to ensure they were infectious?

and if so,

2. May I assume that this included all of the samples listed as "jokes, hoaxes, construction tools, corrupted, intended..." etc. as well?

3. If the answer to question 1 is "no", can you elaborate on what method you used to verify they were infectious?

Thanks for your reply.

I believe certain members of this forum know what i am talking about .. me and some of the rest of the members have some past (in other words, i didn't just appear out of the blue in the vx scene). But in normal circumstances, you're right.

Hi Jim. I didn't think Stavros was your real name, i just thought you'd get the note

Uh, what do you mean with "rooting"?

Perhaps then you should have emailed or IM'd those with whom you have an acquaintance to discuss this since your concern appears to be more of a personal issue between you and other members with whom you are familiar.

Most of the samples, if not ALL of them, have been tested by two friends of mine, who have had the time to test each sample. VMWare was used in most of them, as far as i know.

said by VirusP:

---

Most of the samples, if not ALL of them, have been tested by two friends of mine, who have had the time to test each sample. VMWare was used in most of them, as far as i know.

---

I appreciate your answers to my questions.

Thanks again for your reply.

Hi, your tests are great, they simply confirm what a lot of us already know, only a small group of people really dislike your test, most of them belong to a forum that host's an AV Support Forum that did poorly in your test, the same AV does poorly in all tests but VB's test. Other AV companies did poorly too, but they work on adding detection, rather than come to DSL to trash you your tests or myself and others, who like to look at all tests available.

This is an ongoing battle, any test other VB's will be attacked, trashed and so will the authors of the tests plus anyone that might agree with such tests. Your test shows an AV's overall protection, which is what an AV test should do, The top AV's on your test are also the top on any test posted, they also pass VB's test, which I feel is very weak, but none the less a test.

Now when a person takes every test available and sees where each AV scores they can get a good picture of the AV's detection, and can then decide what AV is the best for them. Also one of the guys that trashed you is the ADM at wilders, claim you don't know how to test and so forth, I hope you got a good laugh, when I posted a response he gave to a poster about his own tests, OH by the way the AV he host won his test, who would have thunk it?

Don't let such be people, distract you from your tests, I look forward to real people doing real tests they have nothing to gain or lose, so the tests are not direct toward one AV over the other. But a AV Support forum test's are useless, why would they say an AV did better than the one they provide support for? they of course wouldn't, they have everything to gain and lose.

I feel the need to copy-paste a little post i made up at wilders. I think it points out my moto for behaving the way i do and telling the things i do sometimes.

"Ok, so since i ain't welcomed, i just have a couple of questions for you:

1) I have been annoyed by the fact that noone from this forum or another, who has been an av specialist-security specialist even-has emailed me, suggesting ways to improve the quality of the tests i perform. The only thing i have got till now from certain av related ppl is discredibility, disapprovement and bad rep.

2) I don't think everybody knows if and how much related certain forums are to specific av software companies .. let's say i got an av software, or work at such a company for all that i care, and start up a nice little forum, praising "my own" av software. Would that be just?

3) Since VB is the best and most credible av testing org in the world, how come they never publish the vx list they use, or the procedure they follow???

4) Why are certain software ONLY being tested at those tests? I managed to gather-up almost 50 (!!!) antivirus and anti-trojan software, how many of them are included in the VB test? Are the rest of them out of the market? Can't a pc user buy one of them? Why are they excluded afterall?

5) Why do i get the feeling that, like in the av market, people in the av scene do NOT want others to "intrude" and learn the game???

6) If i saw some guy trying to learn a job i am pretty good at, i'd try to help him, unless i felt threatend by the fact that one day he could get my job .. i may be considered a "newbie" compared to many of av experts, nevertheless i do the best i can do. What do they do? Sit in front of their screen and start calling guys like me failures. Now, isn't this all a pretty good reason for me to get upset

Best regards to u all

Antony a.k.a. VirusP"

said by JimIT:

---

Were each of the virus samples used in your test executed on a computer to ensure they were infectious?

---

Another--and just as important--question is: Are the samples real threats (i.e. ITW), or are they just some variant that someone hacked up and made available?

I have heard from more than one source I trust that some people have put a whole lot of effort into making dozens of variants of particular malware samples, and then submitting them to selected AV vendors. Obviously, those AV products will suddenly have "better detection", but does it mean anything?

»www.wilderssecurity.com/ ··· sg106502

They don't have to be ITW viruses. Cholera may not be itw atm, but it still is deadly, ain't it?

Some av vendors add variant signatures without adding a whole new virus name. This doesn't change a thing. Anyway, most vendors add it as a new virus name in order to achieve more sales, afterall it's obvious that more virus signatures MANY get more sales.

If every vendor added every zoo malware, then every AV utility would be as CPU intensive and expensive as KAV is.

And ITW status does matter. To respond to your analogy, yes, Cholera may be deadly, but until Cholera becomes an issue where I live, why should I take measures against it? I could take measures--against Cholera and 5,000 other ultra-low-risk diseases--but then my life would be utter hell from all the trade-offs I'd have to make. My doctor's office would be a second home, my medical bills would be significant, I'd have no free time whatsoever, I'd feel like crap all the time, and I'd be at risk of complications. Just like with a "kitchen sink" AV utility, there are serious trade-offs.

As another analogy, until relatively recently, Smallpox was thought to have been totally eradicated, for all practical purposes. Thus, no one was inoculated against it, and no sane person would have argued for widespread Smallpox inoculation. Now that Smallpox may be a potential threat to some (or so we hear), inoculations are being carried out (with some serious trade-offs, including deaths that may be associated with it). Smallpox went from being a "zoo" disease to being an "ITW" (or "potential ITW") disease, so protocol was changed accordingly.

I love KAV (it's actually the only AV I use at the moment), but I try to keep a level head about it. No one is saying that it's not better to detect anything and everything, but if you want that, you'll have to pay for it. And bashing or discrediting products that don't detect ultra-low-risk threats just doesn't make sense, any more than it would make sense to criticize your local government officials for not making widespread Smallpox inoculations available.

That's what all this argument is over: Where do you draw the line between maximum protection and the trade-offs you make to gain that maximum protection? Being on one side of the fence or the other in itself is fine, but what isn't fine is giving some products unfavorable press just because it doesn't adhere to your idea of a favorable trade-off balance!

Round up the 100 top, verified ITW threats (or some other reasonable number), then test AV products against them. Or, if you want to test against a wider range of threats, give tiered results. Report on how each product did on the top 100 threats, on the top 101-200 threats, etcetera, and overall. At least then both sides of the "trade-off fence" would be represented, respected, and the results might actually mean something!

For example, some products will do very well against the top 100 threats, moderately well against the 101-200 threat range, and poorly in the 201-300 range. Some products will do very well against the full 1-300 range. And that's fine--but if reporting was done this way the public would be able to decide what product to go with based on their own "trade-off comfort zone"!

Your post while interesting, is recommending this tester to reinvent the wheel, limit his tests, VB is already the limited test you want, this type of test goes beyond VB testing limits.

If one only wants to see limited testing, then VB tests are the answer for them, but people who want to see tests outside of the box, looks at these tests.

Myself and others want to see the detection rate of the AV on a wide scale. To be honest if one only wants protection against the viruses on the VB test, then don't waste your money on any AV, just get a free AV, and you will be protected against the viruses on the VB test.

quote:

---

...Myself and others...

---

Sofar, you are merely talking on behalf of yourself, Vampofero. Others are quite capable of expressing their own opinion - please don't underestimate people over here.

Although I for one do respect your opinion, seems you are fighting a battle already long time lost.

watcher

I'm not suggesting that he reinvent the wheel. He can include all the same samples; just break up the results (in addition to an "overall" rating).

Yes indeed that sounds like a lot of work, but the work would be determining what the real ITW threats are, it seems. But if you aren't willing to put the work in and determine what the real ITW threats are, to me it seems the test is of limited value. Scientific analysis is a real bitch (trust me, I know; I was a chemistry major).

I don't think that VB100 lives up to the standard I outlined. Go to the VB site, and you see either a "pass" or "fail" rating. They don't go out of their way to explain their methodology, what samples they use, or the fact that a product can fail based on false positives. (Oh, sure--they might reveal that info if you cough up $300+.) They also don't use lesser-risk samples (so far as I know). And I wasn't saying "don't include zoo stuff", but rather, "include zoo stuff, but tier the results so we can see how products do with it, specifically".

To WatchIt...

No, he's (Vampirefo) been talking about me too, and I'm sure others will speak out when they read this thread, unless the thread gets locked or they will be too scared..I think Virus Bulletin is a great test, and yes, perhaps one test SHOULD be held in high regard... BUT, and that's a BIG BUT, VirusP's test does go beyond... I choose to see value in it, just like you choose to see value where you see it...If you can't stand that, I'm sorry...

quote:

---

...(Oh, sure--they might reveal that info if you cough up $300+.)

---

contradicts with:

quote:

---

They don't go out of their way to explain their methodology, what samples they use, or the fact that a product can fail based on false positives.

---

Ergo: the info is available. One can discuss about the conditions, but that's not the issue.

watcher

It doesn't contradict, because the general public isn't privy to the details. If they started posting that information on their site for all to see, then I'd say they were "going out of their way" to make it available. (I didn't say it wasn't available, just that it wasn't easily obtainable.)

Hi, have you looked at ITW list? I mean no disrespect, I am just asking cause most people don't even know what's on it.

A lot of old outdated viruses that wont run on XP, is on ITW list, the viruses on that list is for the most part is no threat, and one is not likely to be infected by them.

True some newer viruses are on the list, but you will see viruses dating back to 1995 on that list also, Do you worry about? I don't here look over this list find how many viruses you feel are not really a threat, now how many are left? this is all the av that get's the award from VB protects one against, not much protection, that's why VB is not much of a test in my opinion.

»www.wildlist.org/WildLis ··· Time.htm

AntiEXE.A...............[D3, New Bug, Ne] 9/94 FpSjSmZz
Form.A..................[Form 18........] 7/94 SmWsZz
JS/Kak.A-m..............[...............] 2/00 AoAsDpEwFpJdJwMoOzPbSjSmSr
ZvZz
NYB.A...................[B1.............] 7/94 SjSoWsZz
O97M/Tristate.C.........[Crown.B........] 4/99 AsFpJwKdSmSoStZbZv
Ripper..................[Jack Ripper....] 4/02 EwSkSo
VBS/Freelink-mm.........[...............] 10/99 JdPhSm
VBS/Haptime.A-mm........[Help...........] 6/01 AoAsAyFpJmJwMsOzPhSaSjSkSm
SoTaZvZyZz
VBS/LoveLetter.A-mm.....[BugFix, I-Worm.] 5/00 AoAsAyEiEwJdJwMsSkSmSoZbZv
Zz
VBS/LoveLetter.AS-mm....[Plan.A.........] 10/00 AoAsAyDpFpMsPhRvSkSoZvZy
VBS/LoveLetter.C-mm.....[...............] 10/00 AoAsSo
VBS/Netlog.A............[Network........] 3/00 AoSmSo
VBS/Redlof.A-m..........[...............] 10/02 AoAsDpEkFpGrJmJwKdMsMtPhSf
SgSjSkSmSoStTaTmWlZvZyZz
VBS/Stages.A-mm.........[ShellScrap.....] 7/00 AoAsFpJdZz
VBS/Tam.A-m.............[...............] 2/01 AoAsFp
VBS/VBSWG.AQ-mm.........[...............] 7/02 AsSk
VBS/VBSWG.J-mm..........[Anna K, Kalamar] 2/01 AoAsEiEw
VBS/VBSWG.K-mm..........[NeueTarife SST,] 2/01 EiEw
VBS/VBSWG.X-mm..........[HomePage, SST..] 5/01 AoAsEiEwJd
W32/Acebot..............[Newbiero.......] 6/02 AoMoSoTm
W32/Aliz.A-mm...........[...............] 11/01 AoAsSmTa
W32/Aplore.A-mm.........[Aphex..........] 6/02 AoAsZbZz
W32/Apost.A-mm..........[...............] 10/01 AoAsTaZz
W32/BadTrans.A-mm.......[13312..........] 5/01 AlAoAsDpFpGrJdKdPhSfSgSrSt
TmZz
W32/BadTrans.B-mm.......[29020..........] 11/01 AlAmAoAsDpEiEkEwFpJdKdMoMs

shoter,

quote:

---

I think Virus Bulletin is a great test, and yes, perhaps one test SHOULD be held in high regard...

---

Reading your comments in the now closed thread, it looked like you expressed a far greater commitment to VB. One can't change sides all the time. You are confusing me - what's it going to be?

quote:

---

..BUT, and that's a BIG BUT,...

---

That's exactly what I'm referring to...

quote:

---

VirusP's test does go beyond... I choose to see value in it, just like you choose to see value where you see it...If you can't stand that, I'm sorry...

---

Actually, in the now closed thread, you didn't see that value in this way. Please don't confuse me, others and foremost yourself. This thread is hard enough as it is...

watcher