Equiinet - NetPilot, CachePilot, SecurePilot (pdf ... - West Coast Labs

ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT 

FEBRUARY 2006 

Equiinet 

NetPilot Plus 

www.westcoastlabs.org

2 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT 

Contents 

Equiinet NetPilot Plus 

Test objectives and scenario ..................................................................3 

Test network ............................................................................................4 

Test methodology....................................................................................5 

Product test reporting ............................................................................6 

Certification ..............................................................................................7 

The product ..............................................................................................8 

Test report ................................................................................................10 

Test results................................................................................................17 

West Coast Labs conclusion ..................................................................18 

Security features buyers guide ............................................................19 

West Coast Labs, William Knox House, Britannic Way, Llandarcy, 

Swansea, SA10 6EL, UK. Tel : +44 1792 324000, Fax : +44 1792 324001. 

www.westcoastlabs.org 


EQUIINET NETPILOT PLUS 3 

Test objective and scenario 

The war for control of corporate inboxes has been raging for some years 

now as Anti-Spam solution providers seek to protect us from unsolicited, 

inappropriate and often offensive intrusions into our time. 

The originators of these emails are becoming ever more inventive and so 

more and more companies are coming to rely on automatic solutions with 

learning engines to protect their users and machines. The emails 

themselves are getting more sophisticated. Spam is now no longer just 

advertising material, but is evolving, and often acting as the precursor to 

identity theft. 

This Technology Report examines the functionality and performance of 

participating Anti-Spam products which are aimed at the small, midsize 

and corporate network environments. It has been open to both software 

and appliance-based solutions plus hosted services. 

The objective of our overall testing program, which is open to all Anti- 

Spam Vendors is, through a real-world test environment, to provide an 

independent validation of Anti-Spam solution effectiveness with particular 

reference to: 

■ A detailed view of the features and functions of the solutions 

■ Spam detection capability and rates of detection of each solution 

■ Integration into a network infrastructure and level of administration 

required to operate effectively. 



Test network 

Software solutions are installed on servers that exceed the minimum 

specifications required by the vendor. 

Appliance-based solutions are installed on the network according to the 

vendor’s recommended placing. 

For hosted services, WCL test through identified email accounts and will 

change the MX records to divert the mail stream through the hosted 

service. In order to allow the DNS change to propagate, service providers 

allow a 2-day settling-in period. 

Details of the tuning and vendor customer support will form part of the 

additional feature testing and reporting. 



Test methodology 

WCL has a number of domains available which act as honeypots for 

spam, receiving genuine, not canned spam. These domains receive 

varying levels of spam and are intended to mirror different email 

environments. 

Within each domain are designated user accounts with a variety of email 

practices and needs - some are subscribed to a variety of newsgroups and 

mailing lists. Some user accounts actively contribute to mailing lists. The 

domain designated for testing purposes will be that which currently 

receives spam at a level consistent with the test requirements. 

For testing in this Technology Report and for the certification of each of the 

participating solutions, we used live mail feeds coming in to various extra 

domains wholly owned and controlled by West Coast Labs. Each domain 

used contains a number of individual user accounts with established email 

addresses, along with distribution lists. 

To maintain the flow of genuine mail, test engineers used several internal 

and external accounts, to send emails that simulated real life email 

transactions common in business: for example requesting meetings, 

sending notifications to groups and non-business related social emails. 

Emails were also sent from web-based accounts to simulate external 

users sending non business-related emails and home workers. Individual 

user accounts were subscribed to several mailing lists and daily 

newsletters for grey mail purposes. 

For each solution we configured the device or software to fit in with the test 

network and placed it into a stream of live mail to see how it would cope 

in an ‘out-of-the-box’ configuration with real-world traffic. However, we do 

recognize that a large part of spam detection relies on an initially intensive 

learning process. Hence, we will be placing these devices in the mail feed 

in coming months for longer periods of time, interactively training them, 

and updating the performance data included in the online White Papers. 



Product test reporting 

For each product that we test, we will issue a report which will address the 

following aspects of the product: 

1. Management/Administration 

■ Ease of Setup/Installation 

■ Ease of Use 

■ Logging and reporting function 

■ Rule creation 

■ Customization 

■ Content Categories 

■ Technical Support Available 

■ Program Help Menu 

2. Functionality 

■ Email Processing Steps 

■ Allow/Blocking of Email 

■ Quarantine Area 

■ Additional functionality reporting 

■ Block Email Addresses 

■ Blacklist/Whitelist 

■ Allow Email Addresses 

3. Performance 

■ Volume or % of spam detected 

■ False positive rate 

■ Spam incorrectly passed through 

■ Legitimate mail blocked 

■ Legitimate subscription mail blocked 



Certification - Checkmark 

Upon successful completion of the catch rate testing, participating 

solutions will be accredited to Checkmark Certifications for Anti-Spam 

subject to achieving the following catch rates:- 

Checkmark Anti-Spam Certification 

PREMIUM – 97% and over Catch Rate. 

www.check-mark.com 

Checkmark Anti-Spam Certification 

STANDARD – 90% and over Catch Rate. 

www.check-mark.com 



The product 

The protection provided by the NetPilot’s SmartUTM is the most 

comprehensive available and includes anti-spam, anti-virus, anti-spyware, 

intruder detection and prevention, advanced Firewall, URL Filtering, email 

policy controls and secure VPN support. For just over 1p per day per user 

(for the average NetPilot configuration) organisations can remove the 

nuisance and time wasting of spam. 

url : www.NetPilot.com 

Equiinet says…about the NetPilot Plus Business Benefits 

NetPilot's anti-spam capability now offers Spam Assessment, SpamCop 

plus, Bayesian Filtering. When coupled with the NetPilot's Email Policy 

controls this adds yet more power and flexibility to defeat spam, 

All new NetPilot Plus and Enterprise units ship with Anti-Spam 

functionality included within the price of the unit. 

url :www.netpilot.com 



The product 

Equiinet says…about the NetPilot Plus Technical Benefits 

NetPilot offers a two layered approach with Anti-Spam controls, with both 

having been integrated to provide common controls and reporting. Firstly 

the NetPilot provides Spam Assessment and Spam Cop together with 

Bayesian learning, giving a powerful combination of standard spam 

countermeasures. In addition, Email Policy Controls can be applied to the 

NetPilot and this can additionally provide quarantine, black and white 

listing, and much greater flexibility for individual organistions to handle 

spam in their specific environment. 

url :www.netpilot.com 



Test report 

Introduction 

The NetPilot Plus device is a compact Unified Threat Management (UTM) 

unit with a sealed fascia that is ideal for sitting on a desktop or on a rack 

shelf in a server room. Indeed, the only components on the front are power 

and disk lights. The rear of the unit contains a PS/2 keyboard connector, 

parallel port, serial connection and VGA connection, and a rocker-style 

power switch. The collection of two onboard NICs plus a further one on an 

expansion card allows for a variety of network set ups to be implemented. 

The documentation arrived as three well written and cleanly styled A5 

sized Getting Started Guides covering various aspects of the device and 

a Resources CD. There were also two large manuals in folders that 

provide a training course in administration of the device. During the course 

of testing, the Equiinet released a new version of the Operating System, 

so the device was upgraded from version 3 to version 4. 

Thankfully, the interface look and feel has not really changed and the 

knock-on effect in terms of the way that Spam is handled is not huge. 

There are, however, significant alterations in other parts of the Operating 

System, for example the firewall component has now been expanded in 

response to user requests and allows a lot more control to be taken than 

was previously possible. 



Test report 

Installation and Configuration 

Initial configuration can be performed either via a PS/2 keyboard and VGA 

monitor plugged into the unit itself; alternately a private network range is 

already set up on one of the NICs and a client machine may undergo an 

IP address alteration if necessary. All interactions are handled via a secure 

web interface – the client may use a standard web browser, and the device 

itself uses the text browser LYNX. 

The opening wizard asks for some basic details to perform the set up. 

Version 4 of the system is capable of performing lookups on both the 

internal network and an external ADSL line. This means that if DHCP is 

enabled the device can be plugged straight in to a network and it can take 

a good guess at how it should be set up. 

In practice this works fairly well, although if the administrator has specific 

IP address set aside for the device, the network settings may need some 

alterations after the wizard has run. Thankfully, these changes are simple 

to find and quick to perform, although for users familiar with version 3 the 

methods of altering the network settings have slightly changed. 



Test report 

The Interface 

Following the preliminary configuration of the device, the SSL encrypted 

web interface is available to devices on the internal network. This has 

retained the clean and attractive look and feel from the previous versions 

but has added in extra options where appropriate. 

Upon initial login to 

version 4, there are 

several shortcuts now in 

place on the front 

screen enabling the 

administrator to quickly 

jump to various sections 

in order to expedite any 

configuration. There is a 

quick link to directly 

alter the network 

connections which then 

links the user through to 

screens where they may alter the behaviour of each interface and it’s trust 

group. This is accompanied by a link to the Firewall overview, with options 

to proceed further and alter firewall rules if necessary, a link to change the 

admin password, and usefully a link to the licensing options screen, which 

was previously buried under the menu structure. 



Test report 

The Interface (continued) 

As the device is multi-purpose there are several sections in the main menu 

available that deal with Firewall, Antivirus and VPN functionality. The 

Spam functionality that is under review here is easy to find with the more 

generic settings being found under the section heading of Email. Policy 

actions can be found under the Email Filter Policy section, and there are 

options here that allow for the creation or editing of policies in some detail. 

Spam functionality is 

enabled as part of the 

wider UTM functionality 

on version 4 or as a 

separate component on 

version 3 via a license 

key system, with keys 

obtainable from 

Equiinet resellers. The 

provided keys need to 

be entered into the 

interface along with the 

hardware serial number 

(found on the rear of the device) in order to benefit from the maximum 

protection that this device can offer. The device then authenticates these 

keys online and activates the protection. 



Test report 


The Spam Filter section under the major heading of Email deals with the 

more universal settings and as such has a link that allows for the quick 

turning on of spam functionality with a single tickbox. There is also a link 

here to set the spam thresholds and to decide what the device should do 

with suspected spam. The ability to use the SpamCop blacklist can be 

turned on or off, and there is the opportunity to reclassify individual mails 

using a MID value from a specialist header that the Netpilot Plus adds in 

called X-Spam-Reclassify. Finally for this section there is the option to 

view some statistics about Spam and genuine mail that has already been 

learnt. The interface also notes that the inbuilt Bayesian spam adjustment 

starts when 200 of each have been through the system. 

The NetPilot plus has 

several options for 

dealing with suspected 

Spam – it is possible to 

deliver it as normal with 

extra headers or to 

quarantine it on the 

device itself. Alternately 

the administrator can 

choose to have the 

message delivered to 

the administrative 

mailbox, either as a 

copy of the original or as an attachment. Although the device adds in extra 

headers to emails, it does not currently allow for the alteration of the 

message subject line to reflect the nature of the email. The device leaves 

it up to the client email program to interpret these and mark them up as 

appropriate, however the manual makes this clear from the outset, and in 

normal corporate usage the quarantine would be used so that messages 

would be unlikely to be delivered to the end users. 



Test report 


The other section for dealing with Spam is under the Email Filter Policy 

section – this allows the user to see how the mail policies are built up and 

alter individual elements of them. There is also the possibility of looking in 

the Quarantine for messages, and most importantly for this section to look 

under the Review and Learn banner to reclassify received mails en 

masse. 

This is achieved via a 

series of tickboxes and 

a drop down box at the 

end of the page. This 

process is as simple as 

the user choosing mails 

that they wish to classify 

as Spam or genuine, 

unticking all the others 

and pressing the submit 

button. In practice this is 

a quick way of getting a 

lot of data into the 

learning system of the device and it works well. 



Test report 


Lastly under the Filter Policy menu is a series of Advanced options. These 

allow the user a lot more flexibility in building policies, and here the 

patterns and actions associated with a policy can be altered. It is also 

possible to configure the alerts and to upload a file to test against the 

policy that has just been built. 

It is worth noting that 

dependent on how an 

organisation sets up the 

NetPilot Plus to deal 

with mail, the Firewall 

options may need 

reconfiguring. This has 

been made as easy as 

possible within the new 

interface and is a quick 

process. 



Results 

Type of Mail Delivered as Genuine (%) Delivered as Spam (%) 

GENUINE 100 0 

SPAM 9 91 

The NetPilot Plus performed well, delivering 100% of the genuine mail 

correctly and correctly classifying 91% of the spam mail in a straight out of 

the box configuration. 

It is also worth noting that the NetPilot Plus solution delivers a good 

proportion of grey and list mail as Spam. This gives an organisation the 

flexibility and opportunity to define policies during a training period whilst 

not delivering mail that could potentially be offensive or defamatory. 



West Coast Labs Conclusion 

The NetPilot Plus offers a good spam engine within an 

excellent UTM device. Although it is fairly reliant on training 

in order to achieve top levels of protection this is a solution 

that can nonetheless be recommended as a remarkable all 

in one package. 

The Equiinet NetPilot Plus device performed consistently 

well in the tests, and therefore West Coast Labs is pleased 

to award the Equiinet NetPilot Plus the Standard level Anti-Spam 

Checkmark. 

West Coast Labs, William Knox House, Britannic Way, Llandarcy, 

Swansea, SA10 6EL, UK. Tel : +44 1792 324000, Fax : +44 1792 324001. 

www.westcoastlabs.org 



Security features buyers guide as stated by Equiinet 

SPAM FEATURES 

Does the product block spam out of box or does it require addition or 

tuning of rules? YES 

Is user feedback required over initial stage of deployment? 

Optional but performance does improve with training. 

FILTERING 

Does the product utilise keyword lists? YES 

Does the product utilise Bayesian filtering? YES 

Can white-lists/black-lists be set? YES 

Does product support RBL? YES 

Does the product support the setting of different confidence levels? Can 

actions be varied at different confidence levels? YES 

Can subject line of messages be altered? YES 

Can email headers be set/amended? YES 

ADMINISTRATION 

Can the product be automatically updated? YES 

Can filters be automatically updated? YES 

What are the update methods? Proprietary 

Can suspected spam be quarantined? YES 

If so, what type of quarantine (forward to Q mailbox / saved on device / 

etc.)? Saved on Device 



Security features buyers guide as stated by Equiinet 

END USER INTERACTION 

Can users see reports individual to them? No 

Can users process messages themselves? YES 

Can users review mail marked as spam? YES 

Can users free messages from quarantine? YES 

Can users set their own white lists/black lists? YES 

ADDITIONAL SECURITY FEATURES 

■ Anti-Virus for email 

■ Anti-Virus for browsing 

■ URL Filtering 

■ Regex Filter with allow and block lists 

■ Email Policy Controls 

■ Advanced Firewall 

■ CIPE and IPSec VPNs 

■ Intrusion Detection 

■ Intrusion Prevention 

■ User and Group Access Controls 

■ Time of Day Controls 

url : http:// 

www.equiinet.com/netpilot/unifiedthreatmanagement/default.asp

Equiinet - NetPilot, CachePilot, SecurePilot (pdf ... - West Coast Labs

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?