Leak tests results (and also a possible bug)

Hi,

I went to firewallleaktester.com and downloaded (or at least tried to) all the leaktests to test them against the ESS firewall module. Here are my results :


Excluded leaktests :

DNSTester - I don't use the DNS Cache service, and advise everyone to disable it too. It's dangerous and not that useful, and can dramatically slow down your connection if you use a big hosts file like the MVPS one.
MBTest - As long as I use a French version of XP, it won't work.
AWFT - installation needed.


Note :

I couldn't download these and since I wasn't able to disable AV-protection (bug*) I wasn't able to test them :

Breakout
CopyCat
FireHole
Outbound
Thermite


Now for the testing :

CPIL - failed, successfully opened a new window with the data and site.
Ghost - failed, sccesffuly opened IE with it's GET parameters.
Jumper - failed.
LeakTest - passed (communication attempt detected).
PCAudit - passed but only because no rule for explorer.exe was defined for general HTTP (as it should be).
PCAudit2 - says it fails, but seems not to send any data.
PCFlank - fails (transmits data).
Surfer - fails (gets its data).
Tooleaky - passes.
Wallbreaker - fails the 4 tests.
Yalta - passes (without the driver, can't install it).


Hence, with only 4 of 11 leaktests blocked (and only because of strict rules), I strongly advise ESET to include an HIPS-like component into their Suite. The firewall is good performance wise and configuration wise even if it takes some time to get used to and could be more ergonomic, but, it needs serious improvements regarding leak test protection.
I know it's still the first Beta and I trust ESET to fix that until release time.

bug* : When, in Advanced mode, I go to the Setup -> Antivirus panel, and try to click on 'disable' either for the mail or real-time antivirus protection, it does nothing. It works only for the Web protection. Also if, in the advanced setup, I try to uncheck the antivirus protection, I can't click OK to close the dialog until I check it back.
Now, does ESS want me to stay protected at any cost ? I think it's either a bug or a design problem, because not being able to disable protection can be annoying.

 

I want to further clarify your results:

4. Yalta:
Failed!
ESS did give a allow/deny prompt , but it was too late - Yalta had already sent UDP packets. However, when "Deny" was clicked, a new rule was created and Yalta was unable to send any further packets.

7. AWFT:
Test 1 - Failed!*
Test 2 - Failed!*
Test 3 - Failed!**
Test 4 - Failed!*
Test 5 - Failed!
Test 6 - Failed!

8. Thermite:
Failed!

9. CopyCat:
Failed!

10. FireHole:
Failed!

14. DNStester:
Failed!

--------------------------
* = Assuming you allow Internet Explorer to connect.
** = Assuming you allow Windows Explorer to connect.


----------------------------------------
Summary of areas where ESS Firewall fails
----------------------------------------

- DLL Injection / Process patching
- Launching under different context
- Timed attacks / PID Changing
- DDE based attacks

 

plz tell us the firewall of nod32 good or not good ??

 

Overall GOOD , but note it is still not finished

 

thanx HiTech_boy

 

I had the same results. I would like to know if ESET will work on this?

 

it seems to be based on Windows Firewall... the same SDK, and overall is not that good.

 

Where did u get that information (SDK part) ?? Source?

 

you have to take beta software for what it is... BETA - it's not finished at all.

The whole point of beta testing is to find problems, then solutions and incorporate them into a beta2..3.. etc, then into a gamma or release candidate (RC). As this process proceeds, I'm sure the failings, and even the non-failing, but "we found a place that needs improving" will get into a subsequent release.

This is beta1 - almost everyone will find something (even a small something) that will be splatted and removed from a subsequent release.

So - is the nod32 firewall good? It's a good start... it needs improvement it would seem, as should be expected at this early beta stage...

 

Yes. As an early (in the stages of development) beta, it's expected not to be completely finished, and to have bugs. However, a beta may (and maybe, should) reflect all the features that will be present in the RC / final release, else it won't be 'good' test material. And to me, an HIPS module is nothing like a little 'fix', and more of a complete feature, that's completely necessary, not only for protection, but to compete against other suites (namely, KIS, which has a very good one) or personal firewalls, which have these modules.
Advanced heuristics may be (and are, as far as I'm concerned) excellent, BUT, they would be nicely complemented by an HIPS. The proof is the failure to pass these tests.

I understand, again, that this is a beta piece of software and new features may be implemented in the future (and I hope so).

I just hope someone from ESET will comment on this, and tell us if such a module will be implemented (or not) in the future beta's / releases.

 

I desagree, HIPS blocks code execution (this a Kaspersky solution), a real firewall blocks communication across your network adapter when a program directly ou indirectly try to access network (like Jetico - a real firewall). HIPS ís a good solution for protection but is not the best solution to fix firewall bugs. Full pass in leaker tests needs to be by firewall specifications, design and operation, not by aditional modules like HIPS.
I dont believe that ESET development team dont know leaker tests, they need work hard to fix it in nexts builds.

 

I disagree with you.

An example of this, and one of the most simple yet effective, is DLL injection. You allow for example iexplore.exe to connect (outgoing TCP) to remote port 80. This is the normal setting, and you can't restrict it much more than that.
Here you inject a DLL that will silently make HTTP connections to (any malicious server). The rule is valid, the .exe is not modified. So ESS -or any firewall that has no HIP capability- will allow the connection.

I didn't test jetico myself, but if it's able to repel DLL injections, it most probably has a HIPS functionality. I talked about Kaspersky because they sell a suite, too, but many firewalls (comodo, outpost, and others) have these kind of capabilities. It is normal and necessary for a firewall (and even more for a security suite) to be able to defend from such techniques. Hence the HIPS (or whatever way they can find to protect against that).

I never said ESET didn't know about leaktests, I only wondered if anyone from them could tell us what they plan to do to correct the failures ESS encounters in its current state.

 

I do trust Eset should fix the problems related with the firewall component and hopefully also implement HIPS if possible.

 

It's a mistaken concept, Jetico make a per process filter in kernel space, by wrapping the TDI (Transport Driver Interface) functionality, intercepting the functions that the applications and/or helper DLLs (WinSock) use to communicate data to and from the transport protocol drivers. This turn Jetico capable to detect dll/driver injection and the majority of mechanisms used by leakers. I dont know Kaspersky per process filter but many others software developers make the per process filter adding a Layered Service Provider (LSP) extension, and inserting it into the LSP chain. The LSP method, however, is not the most thorough filtering solution, as it relies on applications using WinSock to communicate. To bypass the LSP per-process filter, a rogue application would just need to use a driver of its own to communicate directly with the protocol driver through the TDI, thereby bypassing WinSock. Solution for LSP per process filter is add an HIPS module to block executables, dll's, etc. So sorry this is not a firewall, a firewall block communications across network adapter including loopback interface , a real firewall dont block executables, dll's, drivers from load in memory space like an HIPS software.

 

Very well said .

 

But the LSP insertion has other drawbacks, the ones you mentioned but also sometimes problems with the connection (as with IMON, that inserted in the LSP stack and could cause problems sometimes, like strange problems in Opera/Firefox when many tabs were opened, and mix-ups of loaded images, and also slowing down the connection, which is not the case either with KIS or ESS).

Again, I may have been unclear. I know what a firewall -in the strict meaning of the word- is, and what an HIPS is. I was talking about firewall products, i mean, existing software. May them include HIPS modules that are not strictly 'firewall', that's okay with me. My only concern is protection.

I also agree that the firewall is 'a good start' because it's stealthy and once used to the rules creation they're ok, but still, may it be in the firewall or in a separate module, HIPS functionality would nicely complement it to prevent leaks.

 

I agre that's HIPS is a nice complement and increase the security, but in my opinion is not the best way to fix firewall design/operation bugs. I think HIPS is a good tool for advanced users, not for regular usuers, because blocks anything (malicious or not). The metodology implemented by KIS using HIPS blocks leaker launchs and a lot of good applications (more than Windows Vista UAC), not leaker communications.

Talking about Windows Vista we can say "the Vista native firewall have good leaker protection" because blocks the execution around 50% of leakers, but this is mistaken, is not the a firewall blocks, it's a UAC blocks, here a good paper about this: Vista's security impact on the leaktests

I think that a good firewall cannot be dependent of HIPS modules, a good firewall blocks all suspicius communications and leakers are malicious communications related.

 

Yes and no.

I agree that a HIPS or at least a beahavior blocker (such as the one in KIS) can be quite frustrating for newbie users, and also alerts on legit applications. Which is normal, because, as a behavior blocker, it doesn't make a difference between good and bad app, it only warns that the app tries do to something potentially dangerous. Hence it is intended for advanced users who will know when to allow/deny, else it would be pretty useless.

I won't comment about Vista because I don't use it and don't want to use it for several reasons, including the price, the heavy resource usage and the lack of compatibility with most of the applications I use.

Leakers are indeed malicious communication related, but in a more general way, also represent techniques used by malware in general. Some piece of malware can use DLL injection for example to control software I/O and write itself to a USB disk (that's a theoretical example, of course.) or add itself to the registry to start at boot, insert itself into explorer.exe and spread via removable media.
I completely understand your point of view, and I agree that the firewall should not be dependent on a HIPS. However, there are some special cases that can't be blocked but by a behavior blocker (or, of course, an AV detection) Hence my repeated suggestion of a behavioral blocker/ HIPS.
Some will say, you can use an hips like SSM, that's true, I use it myself and it's pretty good and light. But the point of a suite is to completely protect the user in a single product, so for me, it's necessary to include some behavioral monitor/hips component to tighten the security to a maximum. (which could be disabled/not installed by less expert users)

 

I prefer to say normal user and not newbie user, and today many software developers are making changes in your HIPS engines adding application database to not disturb normal users (Symantec for instance), on Kaspersky forums we can find a lot a threads criticizing the only user decision mechanism of HIPS module. Software developers can remember that Security Suites and AV's market share are composed in majority by normal/newbie customers not by advanced/experts users.

So I wait that ESET developers team woks hard on leaker tests blocks to nexts builds, the results posted on this thread shows a poor firewall per processes filter performance on ESS beta 1, but this is normal, it's only the first beta, I hope better results in the future.