WLO

A general comment about the WLO: The Wildlist tries to show which malware is found spreading around the world. “The WildList was a valuable resource, but grew to be more valuable with the creation of WildCore. WildCore is a set of replicated virus samples that represents the real threat to computer users ”. Who gets this WildCore collection is decided by around 10 peoples who mostly work for various big AV companies. I know that some small AV companies do not get the WildCore collections, due the decision of the WLO technical board. In my opinion, this does not sound fair and impartial, because this smaller companies maybe score worser in ITW tests due this. Read more about problems related to the the wildlist on: http://www.people.frisk-software.com/~bontchev/papers/wildlist.html

P.S.: do not ask me for names or company names, if a smaller company wants to comment it, they will do, but not I.
P.P.S.: we also do not get the wildcore collection, but that is not a problem for us and we accept it. but for other av companies maybe it is.

 

Well Alwil, GriSoft and HBED+V are small companies but their reputation isn't that small. I wonder if they can get hands on the samples collection...

 

The main problem of the WildList is not that its collection isn't sent to everybody who needs it. It is sent very widely thoughout the AV industry. The only reason why you don't get it is because you don't have an AV product that you must ensure detects the viruses in this collection.

The main problem of the WL is that it is misleading. That is, viruses which are in-the-wild are not on it and viruses which are on it aren't in-the-wild. (Of course, there are lots of other problems as well, as my paper which you referred to explains.) But the average users have no clue what is ITW and what is not, so they don't realize how bad the WL is.

The WL guys love it, because it gives them a feeling of their own importance. The testers love it, because it's much easier to test detection of 200 viruses as opposed to 200,000. The AV industry loves it, because it is much easier to make sure that your product detects 200 viruses as opposed to 200,000.

The end result is that almost everybody says what a good thing the WL is, while in fact its existance and use are damaging to the users. Oh, well...

Regards,
Vesselin

 

I'm not important. But I would like to second the statements made by Andreas and Vesselin.

 

I wonder if the not-so-good results of e.g. UNA in ITW tests are in some way related to the fact that they probably still do not get the WildCore collection...
They are an AV company and as such they should get it like most others do.
UNA is just an example, could be any other smaller/new company as well.

I agree with all points of bontchev too. His paper about the Wildlist should be put to more attention of the readers, so anyone can read and think about it.

As said, for me it is not a big problem if they do not want to send me it. Anyway in the conditions to get it under point 4, there is a subpoint 4b which says:
b. The recipient is a recognized tester of or is working for an organization that is recognized to test or certify AntiVirus products. [...]
which says that it is not needed that the recipient must be an AV producer (which is under point 4a). But well, I am not concerned about this, I only would like that every company (also the small/new ones) have the same chances to 'fake' test results P) by getting in advance the WildCore collection in order to add detection for that viruses/malware and score better in ITW tests (as the tests are usually done months later).

 

What about independent testers and individual testers? Is there such a thing, or are they just compiling the stats from existing agencies? What are the standards for testing, and who determines them? Who decided what will and will not be looked for? Is spy ware and ad ware looked for?

Thanks
Wildman

 

Hello Andreas,

As far as I know the distribution is also based on trust. If companies or other people do not have certain standards they will not receive the collection which is a good thing in my eyes. Think about if the samples would get sent to anybody that has a so called anti-whatsoever program without any trust model.
Furthermore you complain that it might be a problem that smaller AV companies don't get this collection and will score bad in tests. Well, what about your own tests? How can AV vendors improve in your tests if you don't give your collection to them? Isn't this the same problem? Why don't you start to make the world a little better and help the AV industy and give your testset to them?

@Vesselin
I totally agree with you about the WLO. It's slow, not accurate, etc... But anyway do we have something better?

Bye

 

It is different, as the ITW samples are considered a global ITW and the ZOO samples not. If the ITW samples are considered a global threat that every AV must detect, they should have the same chances. What can happen to send to an AV company a WildCore collection of samples that are already spreading and that most users already get in their mailboxes? Maybe it would be more interesting if no AV company would get the WildCore collection..., so the results would be more interesting. (?)
BTW: the AV industry already has the samples I have (at least nearly all those included in my tests have nearly all samples); they share the samples around, but as I do not have the permission the send out the samples to anyone (for good reasons), I am not allowed to send everything to everyone (and I also do not want to do that in cases I do not trust a company with zoo samples for various good reasons). The companies get the samples from me not because they do not have it already and because they need them to improve, they get it in order to have a chance to check if the results are correct or not.

 

Hello,

What will happen if everybody will have access to the wildlist? The samples will be out of control and bad things happen. As far as I know the distribution of virus samples was always also a matter of trust.

For example if Company X is an antivirus vendor and writes viruses themselves and show the potential customer it is the only AV that can detect this virus would you consider this fair? And would you give such a company access to the wildlist just because it calimed to be an AV vendor?

I assume there are good reasons why some companies get the samples while others don't. There are lot's of people that report and get the samples <http://www.wildlist.org/WildList/200509.htm>. Among them are small but also big companies and also individuals that trust each other.

What do you mean by the AV industry exactly? I am referring to your test: <http://www.av-comparatives.org/>

It would mean that AVG, Sophos, Avast, TrendMicro, H+BEDV and Dr.Web are unable to implement detection for 20.000++ viruses even if they have the files. In this case I would not recommend those AVs.

Or it might be the other case that those or at least some of them do not have access to the samples used in your test. If this is like assumed I might wonder why some companies get the samples while other don't? How do the companies get a chance to improve in your tests if they don't get the files from you?

bye

 

No.

You can ask that companies and maybe they will tell you the truth, which is "yes, we have a backlog of 100.000+ samples for which we still need to add detection" (that true and some of them admitted it me it). But it is more probably that in order to look better they will say "we do not get anything from tester, if we get samples we add them all immediatly". It is in the hands of the companies what they add and what not and to who they trust and send samples and to who not. It is all in their hands. But ZOO samples are not a high priority like ITW stuff, so I would not compare those 2 different things on the same level.

 

This practically means that you share all the missed samples with each AV company you are testing and it's the AV companies fault that they do not add detection for various reasons. Is this correct?

I am just curious because you said that you are not allowed to send samples to all those companies and now you say you are or maybe I get something wrong and you can explain a bit more in detail.

What I got interested in is to figure out if the companies are lazy or they do not get the samples.

 

Even if they do receive the missing samples, some companies simply have not enough analysts to check and then add the missing malware signatures.

Or as IBK has stated, even with these samples, they will always give preference to "ITW" over zoo malware.

 

It is going OT, but anyway:
I would not call it lazyness. They just have other priorities.
You get it wrong I think. Nearly every tested company gets *every* sample from me (as nearly all submit collections), but anyway in the first line the samples are shared within the companies too. So IMO you are right, they do not add for various reasons.

btw, Why you do not register to this forum? I am sure you are a known person from the AV industry, I would be very curious to know who you are. Are you one of the 10 of the Wildlist technical board?

 

Hello Blackcat,

I agree and I also think that some companies do not have the ressources or other reasons whatsoever why not adding detection. But I can hardly believe that this is the case for 9 out of 13 AV companies. Most of those companies are in the business for a very long time so they should be able to hande the stuff.

But let's wait until Andreas confirms that all 14 tested vendors get the samples so we can start to blem the vendors.

bye

 

You should know that I do not want that AV companies get blamed for e.g. reasons like this. That's why I will not tell any names of companies.
And do not forget that if the few companies feel that they do not get enough samples from av-comparatives, they just need to ask the other av companies for their collections and to share with them. If they do not want to do that they can even send me ther collections to me and I will get the permission to send them all samples too.

btw, the topic was about the WLO.

 

Yes, sorry for my inconvenience and anyway it's good that they have priorities after all.

First of all it is a good thing that they get the samples. So nearly all AV companies are not working as they should. I think it would be nice to show in your report which AV companies do not get the samples from you so at least for the few that are not good we know that they have no access to the samples and it's not their fault in full.

bye

 

I do not think so. Because all those do in reality already get 99% of the samples from the other vendors. So they would just use it as an 'excuse' and say that they could be even better otherwise, instead of trying to improve their detection rates like others do. And if they feel it is unfair (which is not) they are free to say that they do not want to get tested.

I re-ask.

 

Now I got confused again because you said that most of the vendors get *every* sample. Why do the vendors have to ask each other instead or sending them *every* sample so they can improve. Isn't this what the tests are for - help the AV companies to show them that they are weak, provide the samples and help them to improve.

Maybe I missed something again but could you please clarify why some vendors get *every* sample while other have to ask other vendors?

I don't think this is important as this is a public forum. As the topic changed a little I just got interested in your testing methods as I do not want to belive that 9 out of 14 vendors aren't doing ok. Anyway thank you for answering the questions so far. I think that will help everybody to understand things a little bit more.

bye

 

Because they usually already get the samples. I said in case they do not get already e.g. the collection of the company xy, they have to ask that company xy.

Of course you do not have to register. I was just curious, as guests usually have something to hide and you defended the Wildlist and tried to bring the topic to other things .
btw, the companies are doing ok. I think you can not say they are not doing ok just because they have other more important priorities.

 

But "usually" is an assumption and you don't know for sure if they have the samples or not.

Let's assume there are three companies (A,B,C). Conpany A and B get *every* sample while C doesn't. So company A get every sample from B and vice versa. But if you send samples from company A to B where is the Problem to send it to company C?

You previously mentioned that they exchange the samples anyway among each other so I just don't get it.

I might risk to say that some companies tell you it is ok to send it to company B but not C? It woudn't be fair play would it?

How does company C know from which vendor to get the files from? It has to contact all other 12 companies?

I didn't really defend the Wildlist I just said that people, not necessarily vendors that need to get on the list are there. Others, vendors in particular either didn't ask to get the samples or there might be good reasons to deny them the files.

 

How do you know this for a fact?

 

like my words man !!!!

 

Hello Stan999,

you can visit their web page and see the list of names there. Almost all vendors are listed there from big to small.

 

this does not mean they get the wildcore; THIS could be an assumptation. why the list of the technical board is hidden and the rules to get it are not displayed to the public? Why e.g. UNA is not listed? e.g. UNA was tested by VB against the Wildlist and they did not score well. A company which product does not detect 100% of ITW viruses will not be considered by users, but no one tells to the public that it is because UNA did not get the WildCore collection in advance. I am concerned about this in this topic and not to the other things you name, which are to distract the attention from the problem of the Wildlist. I mean, if anyone else would have put the attention in this forum about the WLO, you would not have a chance to get OT. But as I had to tell it (because no one seems to know or to care about it) and because I am a'new' tester which does not really like the ITW tests, you (which based on how and what you write I think are someone from the technical board of the WLO) try to get OT; well, just my paranoid thoughts
When I say 'usually' or 'maybe' or 'I suppose' or 'I guess', most times this means 'I know', but I use other words because I am for example not allowed to show all what I know and from who I know it or because I do not want to start a discussion in details. Peoples that read often my posts and know me, know what I mean anyway . In the sentence where I used the 'usually' you can also remove it, the sentence remains valid anyway. But didn't we talk about this already at the VB conference?
BTW, av-comparatives may be not perfect too, but do we have anything better? (for sure, I just joking; at least it is a bit more transparent than the Wildlist).

Anyway, the topic was discussed a bit, and it was quite interesting. At least now some forum readers know more than before, and that is good.

 

I have yet to see anyone answer my post #6. Is this not the appropriate thread? If not I can ask again in another thread! I am one of the uneducated in this category, so please enlighten me. I occasionally see lists named and sites named about testing, and individuals state that protection programs are good or bad based upon these test, so that is why I asked my questions in post #6.

Thanks
Wildman