Security → Informal Trojan Detection Tests

Hi All:

Curious about how well anti-virus and anti-trojan applications perform against a well known Remote Administration Trojan (RAT) like Sub7? I was. In fact, I was especially curious to see just how effectively popular anti-virus and anti-trojan application would perform when confronted with trojan servers that had been packed with any one of the many executable compression programs available on the Internet.

So, I ran some tests. I used the Sub7 2.13 MUIE server, packed several different ways (and even left completely unpacked in one case). I selected seven programs to test: four dedicated anti-trojan programs and three standard anti-virus programs. I also decided to do more than just scan files on the hard drive, so I ran tests to check the performance of these applications' real time monitoring components.

The results of these tests are detailed on this page:

»www.staff.uiuc.edu/~ehow ··· ests.htm

Before you jump to any hasty conclusions based on what you see below, please take the time to read the Disclaimers & Limitations section at the end of that document.

I hope you find these tests interesting and useful.

Best,

Eric L. Howes
eburger68@yahoo.com

said by eburger68:

---

I hope you find these tests interesting and useful.

---

Very much so Eric--thanks. I'm sure some of my DSLR colleagues will have suggestions to refine the tests, but it's probably safe to say we all appreciate the effort you put into them and the fact that you're sharing the results here.

said by eburger68:

---

Before you jump to any hasty conclusions based on what you see below, please take the time to read the Disclaimers & Limitations section at the end of that document.

I hope you find these tests interesting and useful.

---

Looks like you went to a lot of work testing everything. I've been considering TDS-3 and it's too bad the trial version does not allow you to test Execution Protection.

I found your info to be very useful and am giving it a Thumbs Up!

Thanks for the info.

BOCLEAN ????????

I love the thinking processes you put me through! Now I have to go back and rethink what changes, if any, I want to make to my system. There is some really good, significant info in all of your testing, obviously. Something there for everyone/anyone to benefit from. As always, thanks for all the work you do and that you share with us.

I am impressed, the tests were methodical and well-described. As you point out, the testing is limited in what it tells us, but just as it is, it's one more useful yardstick for users to measure by.

What I would add to the testing first and foremost are two things:

1. A standard set of trojans numbering perhaps ten, chosen for their popularity and insofar as possible, for their variations in methods of stealth, etc.

2. I would use a few well-defined methods to deliberately attempt to bypass the detection methods of the anti-trojan apps. In particular, wherever the trojan allowed it, I would configure for various restart methods _and for none at all_. (An intruder doesn't need to risk detection by placing an obvious entry in the "Run" key. He just needs the trojan server to start up and open up access for a few minutes. Once in, he can set up persistent execution however he wishes, while disabling or circumventing countermeasures at will.)

Also, as ZZZZZZZ has pointed out, BOClean merits inclusion in the test.

Because BOClean is not a file scanner and operates in a different and more system-involved manner than other A-T apps, it often gets short shrift in comparison tests. It does its real work only when the trojan is actually executed, so it requires somewhat different testing methods. I think it deserves better consideration.

If you plan additional efforts along this line, Eric, I would be interested in helping out.

pchelp

said by pchelp:

---

What I would add to the testing first and foremost...

...I would be interested in helping out.

---

Well... just pick out something to do and do it. I couldn't help but notice how well Lockdown performed.

said by dkoert:

---

said by pchelp:

---

What I would add to the testing first and foremost...
...I would be interested in helping out.

---

Well... just pick out something to do and do it.

---

Not sure what you mean by this. I could assist with testing suggestions based on my experience, with copies of actual trojans and the choice thereof, and with remote accesses if that were part of a test procedure.

I'd be interested in a collaboration as opposed to working alone. It's more efficient for one thing.

quote:

---

I couldn't help but notice how well Lockdown performed.

---

I suppose you're referring obliquely to my past criticisms of Lockdown 2000.

For the record, the Lockdown Pro and Lockdown Millennium products are not the same as Lockdown 2000 and were never at issue; neither in my statements nor in the ongoing libel lawsuit.

It appears to me the Lockdown 2000 product has been abandoned.

Well done.
Just the kind of info I need to try'n secure my system.
Been looking in a lot of places on 'de web' to see what other folks are using (sure are a lot of choices!) to find something I can use that I can have a bit of faith in before I start.
This will Definitely help me narrow it down!
Thanks for your time & effort.

To all who responded:

Thanks for the thumbs up. It was an interesting set of tests to run, and I'm glad you all find them worthwhile.

pchelp:

You wrote:

said by pchelp:

---

What I would add to the testing first and foremost are two things:

1. A standard set of trojans numbering perhaps ten, chosen for their popularity and insofar as possible, for their variations in methods of stealth, etc.

---

Yep, I really do need to expand the set of trojans being tested. As this was the first test of this nature that I had run, I was more concerned with keeping things manageable and under control. Even with one trojan (and a handful of variants), things became quite complicated in a very short amount of time. It was a learning experience.

I'm always open to suggestions for specific trojans to test (and for compression methods, etc.).

said by pchelp:

---

2. I would use a few well-defined methods to deliberately attempt to bypass the detection methods of the anti-trojan apps. In particular, wherever the trojan allowed it, I would configure for various restart methods _and for none at all_. (An intruder doesn't need to risk detection by placing an obvious entry in the "Run" key. He just needs the trojan server to start up and open up access for a few minutes. Once in, he can set up persistent execution however he wishes, while disabling or circumventing countermeasures at will.)

---

All excellent suggestions. Again, I'd be interested to hear a few ideas along these lines wrt specific trojans and bypass strategies.

said by pchelp:

---

Also, as ZZZZZZZ has pointed out, BOClean merits inclusion in the test.

Because BOClean is not a file scanner and operates in a different and more system-involved manner than other A-T apps, it often gets short shrift in comparison tests. It does its real work only when the trojan is actually executed, so it requires somewhat different testing methods. I think it deserves better consideration.

---

I am eager to try out BOClean, but there's no trial version available. Milly, over in the GRC groups, suggested approaching the BOClean folks about obtaining a trial version for testing purposes, but I haven't gotten around to it yet. I have heard good things about that app, though.

said by pchelp:

---

If you plan additional efforts along this line, Eric, I would be interested in helping out.

---

I anticipate doing another round of tests in a few weeks. (The start of the new semester is tomorrow, so I've got a few things on my plate before I can return to this issue.) As for apps to test, I've already identified ANTS and Trojan Hunter (two increasingly popular anti-trojan apps), McAfee Virus Scan and Norton AntiVirus (two widely used anti-virus apps), and AVG (a popular, free anti-virus app). I'm always open to other suggestions, of course.

Truth be told, the biggest question on my mind at this point is the whole situation I described in the second memory scan test for AVP. I really would like to find out what kind of protection AVP Monitor would provide when a cracker or script kiddie attempted to connect to a server that had been loaded before AVP Monitor was started.

Please feel free to throw any other ideas you have for potential trojan detection trials my way. I can't promise that I'll be able to act on them immediately, but I will definitely consider them.

All the best,

Eric L. Howes

said by eburger68:

---

said by pchelp:

---

What I would add to the testing first and foremost are two things:

1. A standard set of trojans numbering perhaps ten, chosen for their popularity and insofar as possible, for their variations in methods of stealth, etc.

---

Yep, I really do need to expand the set of trojans being tested. As this was the first test of this nature that I had run, I was more concerned with keeping things manageable and under control. Even with one trojan (and a handful of variants), things became quite complicated in a very short amount of time. It was a learning experience.

---

Yes, in fact the more I think about it, the more inclined I am to suggest something more like half that number of trojans.

Many of the hundreds of trojans floating about are closely similar to one another. A representative sample that includes several of the more widely-used while also including differing protocols, stealth tactics, etc., could probably be assembled; numbering perhaps five or six.

When one considers the number of combinations and permutations necessary to testing, it quickly becomes a problem as the number of trojans rises. Given several compression options, config and startup options, perhaps one or two droppers or installation strategies, tests of removal, etc., the numbers start reeling up. To test a dozen aspects of ten trojans requires 120 individual tests, each involving some preparation and followup.

quote:

---

I'm always open to suggestions for specific trojans to test (and for compression methods, etc.).

---

Certainly SubSeven, BioNet and BO2K. All popular, and each has aspects of stealth or behavior that makes it important to test. SubSeven has options for startup, BioNet sabotages firewalls, BO2K hides from process monitors. Beyond that I'd want to look at the current field of candidates. It's always changing.

BTW, There are certain capabilities I have been expecting to see emerge in trojans for quite some time, most of which apparently haven't surfaced. For instance, I had expected more trojans to adopt the tactic of BO2K for foiling process monitors.

quote:

---

said by pchelp:

---

2. I would use a few well-defined methods to deliberately attempt to bypass the detection methods of the anti-trojan apps. In particular, wherever the trojan allowed it, I would configure for various restart methods _and for none at all_. (An intruder doesn't need to risk detection by placing an obvious entry in the "Run" key. He just needs the trojan server to start up and open up access for a few minutes. Once in, he can set up persistent execution however he wishes, while disabling or circumventing countermeasures at will.)

---

All excellent suggestions. Again, I'd be interested to hear a few ideas along these lines wrt specific trojans and bypass strategies.

---

As and when needed, I can certainly offer a list of such ideas.

quote:

---

said by pchelp:

---

Also, as ZZZZZZZ has pointed out, BOClean merits inclusion in the test.

Because BOClean is not a file scanner and operates in a different and more system-involved manner than other A-T apps, it often gets short shrift in comparison tests. It does its real work only when the trojan is actually executed, so it requires somewhat different testing methods. I think it deserves better consideration.

---

I am eager to try out BOClean, but there's no trial version available. Milly, over in the GRC groups, suggested approaching the BOClean folks about obtaining a trial version for testing purposes, but I haven't gotten around to it yet. I have heard good things about that app, though.

---

I'm acquainted with Kevin McAleavey, creator of BOClean, and I'm confident he'll be interested and cooperative. He's a very forthcoming and knowledgeable individual.

quote:

---

said by pchelp:

---

If you plan additional efforts along this line, Eric, I would be interested in helping out.

---

I anticipate doing another round of tests in a few weeks.

---

In mid-February I expect to be deposed in New Hampshire, where I may spend as much as a week. After my return to Washington State would be an ideal time for me.

quote:

---

(The start of the new semester is tomorrow, so I've got a few things on my plate before I can return to this issue.) As for apps to test, I've already identified ANTS and Trojan Hunter (two increasingly popular anti-trojan apps), McAfee Virus Scan and Norton AntiVirus (two widely used anti-virus apps), and AVG (a popular, free anti-virus app). I'm always open to other suggestions, of course.

---

All fine choices. I'd have recommended ANTS. Subjected to extensive testing and very widely used, McAfee and Norton are known quantities, and they'll serve as a virtual standard for comparison. The question in the minds of a large proportion of home users will be, "Will it really help to use additional forms of protection beyond [McAfee/Norton]?"

quote:

---

Truth be told, the biggest question on my mind at this point is the whole situation I described in the second memory scan test for AVP. I really would like to find out what kind of protection AVP Monitor would provide when a cracker or script kiddie attempted to connect to a server that had been loaded before AVP Monitor was started.

---

A very good question. If it's something AVP doesn't spot on a scan, I think there's not much hope. But if it is recognized - ?

quote:

---

Truth be told, the biggest question on my mind at this point is the whole situation I described in the second memory scan test for AVP. I really would like to find out what kind of protection AVP Monitor would provide when a cracker or script kiddie attempted to connect to a server that had been loaded before AVP Monitor was started.

Please feel free to throw any other ideas you have for potential trojan detection trials my way. I can't promise that I'll be able to act on them immediately, but I will definitely consider them.

All the best,

Eric L. Howes

---

The most interesting tests I have seen yet, was done by Mr.Paris, and he proved just by packing Sub77, AVP would let Sub7 do what it wanted to do to your PC, and AVP wouldn't as much as blink, Now that was a test.

I would like to see the results, of every scanner, that took Mr.Paris test, I found it quite powerful. »Re: Lockdown Online Security Tests

Nice job. Excellent.Thank you.

said by Vampirefo:

---

The most interesting tests I have seen yet, was done by Mr.Paris, and he proved just by packing Sub77, AVP would let Sub7 do what it wanted to do to your PC, and AVP wouldn't as much as blink, Now that was a test.

I would like to see the results, of every scanner, that took Mr.Paris test, I found it quite powerful. »Re: Lockdown Online Security Tests

---

Perhaps you didn't read Mr Howes' report. He did much the same test, except much more comprehensively and against a long list of anti-malware applications, not just one AV scanner.

Here's the link again:

www.staff.uiuc.edu/~ehowes/trojans/tr-tests.htm

I read the tests at that link, and Mr.Paris test was much more involved than this person tests, sorry, I see the difference and the impact and you don't. The tests are totally different and Mr.Paris test is on top.

Mr.Paris took a Trojan, that AVP could ready recognise and packed it, then ran the same Trojan and AVP just stood there with egg on it's face, the tests here don't compare to that at all.

said by Vampirefo:

---

I read the tests at that link, and Mr.Paris test was much more involved than this person tests, sorry, I see the difference and the impact and you don't. The tests are totally different and Mr.Paris test is on top.

Mr.Paris took a Trojan, that AVP could ready recognise and packed it, then ran the same Trojan and AVP just stood there with egg on it's face, the tests here don't compare to that at all.

---

I don't wish to be contradictory, V, but you evidently haven't understood.

Using the same trojan and the same -- and more -- methods of compression, Mr. Howes has done the same test as Paris, but he did it on seven applications, not just two.

Mr.Paris described his simpler test in a narrative fashion. But if you'll read carefully, you'll see that Mr. Howe did the same things and more.

Howes' results were substantially identical to Paris'. But they provide a much broader base for comparison.

"Comments: AVP missed only one server: the server compressed with ASProtect. This performance puts it near the top for this particular test, along
with TDS-3."

Precisely as in Paris' demonstration, Mr.Howe found that the server packed with ASProtect was unrecognizable to AVP. But placed in perspective with other apps, the performance of AVP was comparatively good.

Notably, ASProtect is specifically designed to counter applications that unpack/decrypt executables or "dump" them from memory. Check out its web page:
www.aspack.com/asprotect.htm

Merely packing the executable (as opposed to "protecting" it with ASProtect) did not fool AVP in either test. ASProtect does its job well, evidently. And just as evidently, so does AVP.

Just as I am, Mr.Howes is concerned about the potential for malware to hide from AVP's memory scans. We are also both interested in exploring the matter further.

pchelp

Very impressive Eric. You've done a very good and objective analysis. A few thumbs up for your efforts.

I just wanted to also say thank-you for an informative report. I am not surprised by the good marks for TDS-3. It would be nice to see a comparison with the EXEC protection enabled. I purchased the product and still have not even scratched the surface of what it is capable of. It is quite resource intensive so I don't run it in the sys tray so I don't believe the EXEC protection will function. The have a script available through the forum that adds a right click context menu option for scan on demand. This is planned on being included in the next release.
I believe the Trojan writers are becoming more clever with products like this available, it will only be a matter of time until something similar to BioNet is capable of bypassing any and all detection applications. The only course of action is to be digilant in what code you allow to run on your system and keep good backups in case of infection.

Thanks Eric

said by eburger68:

---

I am eager to try out BOClean, but there's no trial version available. Milly, over in the GRC groups, suggested approaching the BOClean folks about obtaining a trial version for testing purposes, but I haven't gotten around to it yet. I have heard good things about that app, though.

---

I asked Nancy McAleavey [nancymca@privsoft.com], CEO of Privacy Software, for a demo copy of BOClean to use for comparative evaluation and she said no can do. However, she pointed out there is a money-back guarantee so you could get a refund if you were not happy with it.

I would be interested to see how Trojan Hunter »www.mischel.dhs.org/troj ··· nter.asp compares.

Thanks for the hard work you put into this!

Eric,

In case you do need specific trojans to perform some more tests, you could drop me an email on webmaster@wilders.org

You mentioned ANTS for testing purposes; Within a week or two a fully redesigned version 2.2. will be available (for free) at our site, being a full English version as well. I'm more then happy to provide you with a copy as soon as it is tested and released.

FYI: a (rather small) test using TDS-3/Licensed, having execprot enabled, shows for the moment TDS-3 passes the test with flying colors.

regards.

Paul Wilders
webmaster »www.wilders.org

Wingman:

You wrote:

said by wingman:

---

I asked Nancy McAleavey [nancymca@privsoft.com], CEO of Privacy Software, for a demo copy of BOClean to use for comparative evaluation and she said no can do. However, she pointed out there is a money-back guarantee so you could get a refund if you were not happy with it.

I would be interested to see how Trojan Hunter »www.mischel.dhs.org/troj ··· nter.asp compares.

Thanks for the hard work you put into this!

---

That's disappointing to hear. Oh, well. I do plan another round of tests in a few weeks. Trojan Hunter will most definitely be included in those tests, so stay tuned.

Best,

Eric L. Howes

Paul:

You wrote:

said by davidovv:

---

Eric,

In case you do need specific trojans to perform some more tests, you could drop me an email on webmaster@wilders.org

You mentioned ANTS for testing purposes; Within a week or two a fully redesigned version 2.2. will be available (for free) at our site, being a full English version as well. I'm more then happy to provide you with a copy as soon as it is tested and released.

FYI: a (rather small) test using TDS-3/Licensed, having execprot enabled, shows for the moment TDS-3 passes the test with flying colors.

---

Thanks for your several offers of help. I'll keep them in mind. I am eager to test ANTS (as well as Trojan Hunter), as those apps seem to be recommended more and more around here.

I'm not surprised that TDS-3's Execution Protection performed well for you. Wish I could have tested it myself, though.

All the best,

Eric L. Howes

Rocktagon, Wildcatboy, and the rest who have replied:

Sorry to take so long to get back to you all. The new semester started today AND I got a new system, so I was sans net connection for almost 24 hours while I set it up (quite a difference from the old clunker on which I ran all those tests).

It's very heartening to hear that you all found the tests worthwhile and helpful.

When I get around to doing another round of tests in few weeks, I'll post an announcement here.

All the best,

Eric L. Howes

pchelp:

You wrote:

said by pchelp:

---

In mid-February I expect to be deposed in New Hampshire, where I may spend as much as a week. After my return to Washington State would be an ideal time for me.

---

Thanks for all your insightful comments and suggestions. It'll take me a while to get the start of this new semester sorted out, but when I do get around to designing a new round of tests, I'll be in touch.

Best regards,

Eric L. Howes