DrWeb and NOD 2.50.2 heuristics!

LowWaterMark - This thread got rather heated and included many comments that were either overly hostile or directed at posters, not the test itself. To be fair, all hostile and personally directed comments were snipped out and replaced with a simple "~~~~". A few posts that literally were nothing but hostile or personally directed remarks were deleted entirely. No words were added, only cut. No post was deleted if it had comments on the test itself, only if it was entirely personally focused.

I'm sure no one will be happy as virtually everyone had something cut, but I believe I cut fairly across all posters and for the same reason. This was the only way to return this test and the basic discussion about whether these types of tests have value and if so, how much.

Other than to add this section of red text, this first post by Firefighter was not otherwise altered.



I don't know about you others if you are able to make a pure heuristics scan with DrWeb but I have managed to do that yesterday somehow.

1. First I had to shut down the internet connection. Removing all VDB files from DrWeb folder except the "drwrisky" one, I can get only those "risky" samples by disabling heuristics.

2. I collected those risky samples to a new "risky" folder.

3. After that I turned the heuristics scan on and scanned all the rest samples I have by drweb.

4. I replaced that "drwrisky" VDB file with "drwnasty" VDB file and scanned that new "risky" folder by Drweb with heuristics on. So I have scanned all my files by heuristics ONLY.

My results with DrWeb and NOD against my 3665 infected archived samples.

I don't know what does that cause when all my samples were archived ones but still.

Btw, can anyone say how the heuristics ONLY tests can be allowed with BitDefender, McAfee, F-secure and Kaspersky for instance?


Best regards,
Firefighter!

 

I'm also interested in AntiVir and AVG heuristics if you could somehow test them...

 

Tried with AntiVir but no success.

Best regards,
Firefighter!

 

ArcaVir?

 

LOL

Can somebody enlighten me please (actually i'm very dumb) why should joke viruses be detected via heuristics?

I'm willing to learn from Firefighter

 

I'm not sure this test really has value for anyone picking an AV.

 

Good heuristics can help detect zero-day infections before definitions are available and the end user updates their AV.

No matter how fast definitions are provided for a new infection and the end- user updates there can still be a one to three hour delay at best before they are protected.

Another good test is the Retrospective/ProActive Test provided
by Av-comparatives.
http://www.av-comparatives.org/forum/index.php

 

Why would tests need to show some value to select AV? Haven't you ever asked yourself how effective could heuristics be without any signatures? Well,i did and this one looks pretty interesting to me.

 

Of course heuristics are important. And nobody complains about making tests to determine the heuristic strengths of antivirus engines.

~~~

There is absolutely no reason to pick up a joke "virus" via heuristic detection!
~~~

 

You need heuristics to tell a bad joke from a good joke?

(Sorry! Now that was a bad joke!)

EDIT: Thanks for all the time you dedicate to making tests, Firefighter! It is interesting to see how the different AVs perform...

 

Happy Bytes,none of AVs detected any Joke program so that part is a draw and we can exclude that entry in our mind, ok?

 

I believe that is just FF's standard classifications for his 3665 test files. I don't see any reason for him to have to rework his table just for this test.

I, for one, appreciate Firefighter providing this and the amount of work involved.

 

I think ~~~ people do not understand why such ~~~ tests are useless. Regarding detection of NOD32 heuristics i wouldn't have any reason to complain - it's the winner in this test with enabled AH!

But this test is still flawed ~~~! Instead of testing joke viruses, polyengine creators and all such crap he should test heuristics for what it was designed for - detecting new CURRENT threats such as new mass-mail worms, spammed trojandownloaders and the like!

This is MUCH MORE IMPORTANT to know for the enduser how a heuristic scores there. Numerous of Mytob Worms, Sober Worms, Bagle Downloader Trojans PROVING THIS

 

~~~~~, I think that this test already illustrates a lot For Us about heuristics. Maybe not in the way You Think is correct, but for us it's ok.

 

Let's see what a professional av tester (Andreas Clementi, IBK) says to this.
I'm pretty sure he will comment on this test.

 

Wouldn't that have been easier to say to start with ~~~~?

The only thing I know about AV testing is what I've read and absorbed from others. I also know that I don't have the time, resources, knowledge, or desire to perform a professional test myself; I'd assume that is the same for many members here.
My point is, I think Firefighter's willingness and desire to learn and test should be encouraged and constructively criticized by others more knowledgeable... ~~~

 

that's a hard one

 

ok, i agree to Happy byte to some point. But looking back to Firefighter post, i realize he never gave his opinion on the test nor did he gave the conclusion to the test. ( Some people argue results speaks for itself. )

Firefighter test Heu... may be because he... likes doing it? ~~~~~ All he did was show how current heu works. It doesn't detect "Joke" program like you said. And that is something i learn today.

Firefigther ( And others ) never said NOD32 heu was bad. in fact properbly 99% of people in this fourm think NOD32 heu is the best in the world. And all of use know no Heu is perfect!

Ok. so for newbie it may be misleading. But so is every marketing /advertisment to the world dogs.

So i think firefigther is only sharing his knowledge, and he deserve a little more; respect perhaps? ~~~~

 

Regardless of his manner (maybe not enough / or too much coffee):

He is 100% correct =

<
I think ~~~ people do not understand why such ~~~ tests are useless. Regarding detection of NOD32 heuristics i wouldn't have any reason to complain - it's the winner in this test with enabled AH!

But this test is still flawed ~~~! Instead of testing joke viruses, polyengine creators and all such crap he should test heuristics for what it was designed for - detecting new CURRENT threats such as new mass-mail worms, spammed trojandownloaders and the like!

This is MUCH MORE IMPORTANT to know for the enduser how a heuristic scores there. Numerous of Mytob Worms, Sober Worms, Bagle Downloader Trojans PROVING THIS
<

 

It isn't the fact that he is correct or not ~~~~~.

~~~~~~~~~~~~~~~

Also, as a long time NOD user I would like to see an Eset mod post in bit more friendly fashion. That is just my personal take.

I guess, I would also have to put myself in that group that doesn't think the test is completely useless but does provide some valid limited information as to heuristic detection.

 

Do you think it is fun for me to repeat again and again that such tests are flawed? Take a look to over here:

https://www.wilderssecurity.com/showpost.php?p=426576&postcount=57

and please do not tell me it's unprofessional explained.

And just a note: I did ask firefighter for a FEW of his samples to take a look at it IN A PROFESSIONAL WAY, WITH A DISASSEMBLER - not just scanning them with other AV.

Refused! He told me then he has to build up his test set again!
And now i tell you one thing: Verifying this samples takes much longer than just building a new testset - beside of this i did only ask for a FEW samples and not all. Build up your own mind WHO acts here unprofessional.

 

C'mon,so what if few samples are not exactly what they suppose to be(or to be in the testbed at all).
All AVs get the same testbed to process. And we're not making any conclussions based on Firefighters tests. But it's always interesting to see them. And from my not so small knowledge of AVs,all results are pretty much as they were expected. And they are usually very similar to those performed by professionals like IBK. So why bashing? I found FFs tests to be very useful. We all saw nice detection boost for NOD32 in FFs tests when ESET upgraded AH to detect trojans. Sounds perfectly acceptable to me.

 

Hey Happy Bytes,

Because you are an Eset mod representing Eset, while preaching to the choir on a public message board your posts are being read by existing NOD and potential future NOD customers. Because of that I think the extra effort is worthwhile to keep the posts as professional as possible ~~~~~~.

Just my opinion.

 

My opinion was asked, so here is my opinion: altough I usually accept any kind of test in some way (as also the bad ones can show something), I do not like tests bad tests that does not show something useful or flawed. Of course the time and effort spent by FF has to be appreciated anyway, as he did it with his best knowledge that he has in doing this. But that is not enough. Please understand also my POV when I say that such tests (FF is not the only one and my comments are not always direct only to him) are misleading and should be avoided; the reason is that such tests get much popularity by peoples that think that the test is good even if it is not. AV companies are really unhappy about unprofessional tests, you may think it is because they score bad, but that is not the reason - it is because they are useless and bring wrong facts public; that's why also companies that score really good in such tests are against such tests. On the other hand many professional peoples and organizations are working many hours each day in order to provide good, unflawed and useful tests, and of course they feel a bit frustrated when they see that some users pay more attention to a 1 hour test which does not really show much (this is not the case just at FF). I also got critic in past from some av companies and others about how to perform tests, and we worked to find a solution on how to perform good tests that are useful and unbiased. In some way I find it funny that some readers do accept such tests and some others have problems to understand and accept some others tests where more time and efforts of professionals were spent to provide professional tests . Unfortunatly it seems that they are not reading the wilders forum (maybe that's better ;-)).
Other thing: I read that some of you are bashing HB just because he is Moderator in the ESET forum. I think you all should understand that just because someone is moderator the person behind still has own opinions and what the person does express has nothing to do with what the person does, is working for, or which status the person has in a forum. I for example was a bit "attacked" just because I had the "AV Expert" under the nickname . I know HB a bit, sometimes his posts may sound (or be?) a bit hard, but that is just his way to express his opinion. And about his skillness: I know that HB worked for a lot of AV companies and is probably one of the guys with most professional knowledge in the AV area (which would explain why so many companies would like to get him).
@FF: please try again the CRC thing, I really want to help you to improve a bit your set. Just PM me again, if it really does not work we will have to find another tool or something.

 

I wrote to you but you still missed the point.

As you may already know, this is my private "children's sandbox". After that I've sent some samples to some av-vendor, I have to start making a new collection again, which I don't like to do. That's because after that my samples are already known and not RANDOMLY picked ones. If you are thinking this a bit, you understand.

As we see now, you didn't understand this.

Best regards,
Firefighter!