Download technology report (pdf, 580k) - West Coast Labs

Download technology report (pdf, 580k) - West Coast Labs


Anti-Virus Firewall Solutions

An Independent Technology Report produced by

Product Testing, Evaluation

and Certification Services

West Coast Labs provides a superior

quality testing and certification service for

infosecurity technology developers and has

established independent industry-accepted

standards on product effectiveness and

performance for the benefit of corporate

end-users and decision-makers alike.

Through its global reach, West Coast Labs

brings technology developers and corporate

end-users together, creating a meaningful link

between what the market needs and what

technology developers are offering.

West Coast Labs Services

■ Advanced product testing and


■ Product feature and performance


■ Product-design review and


■ Beta testing and evaluation

■ Custom testing

■ Certification

■ Marketing your technology

message to a global buying market

For full details of West Coast Labs' product testing, evaluation and certification

services contact Mark Thomas, Sales Manager:



Blended threats need to be

addressed by a unified

response for greater security


Simulating realistic business

processes is essential when

testing product capabilities

Welcome to the

second of West

Coast Labs

Technology Reports. The

primary focus of this issue is

Anti-virus Firewall technology.

Part of the Haymarket

Publishing Services Group,

West Coast Labs’ is a wellestablished

and leading

Jon Stearn

CTO, West Coast Labs

independent testing facility for

information security products and services.

Working with over 60 of the world’s leading technology

developers, it has a reputation for high standards of

testing and objective judgement of the effectiveness of

product performance and functionality

West Coast Labs provides leading edge testing,

evaluation and certification services and, based on its

tests of leading IT security technologies, it is able to

offer up quantitative data upon which sound

management and purchasing decisions can be made.

The knowledge base of Technology

Reports, White Papers, Custom Test Reports,

Certification Results receives over 500,000 hits a

month, a clear indication of the value that security

professionals put on this resource and the regard they

have for it as a research tool.

Unified Threat Management

In recent years, the rise of UTM solutions has

transformed the security market. Instead of single

function appliances and services, developers are

increasingly offering products which deploy multiple

security features in a single solution, allowing users to

achieve high levels of security with flexible, easily

managed solutions.

Late last year, the Yankee Group reported that

“…firewalls combined with Anti Virus are the two most

highly valued security solutions”, helping customers

thwart blended threats by offering a variety of

functionality and performance benefits.

This AV Firewall Technology report looks at

appliances from Juniper Networks and Equiinet, plus a

managed service provider - SecurePipe.

The full test results are available online at

The overall objective of this test was to evaluate

each AV firewall product in a controlled

environment. Throughout the test period, each

product had internet access and was configured to

update online as recommended.

The testing environment mirrored that of a small to

medium sized business or branch office: the internal

interface of the firewall was connected to a 100Mbs

network, and traffic loads were set accordingly.

The products were tested in accordance with the

functionality criteria set out below, which form part of

the Checkmark certification programs for Firewall Level

1 and Anti-Virus Level 1. See

The White Paper test reports online address three

specific areas: firewall competency, AV detection

functionality, and performance testing.

Outline test specifications

Firewall competency

A range of tests were carried out using a variety of

firewall scanning tools. These were configured with full

knowledge of both the firewall and network


Tests were conducted to confirm that:

■ All specified outbound services (and no others)

were available from internal clients.

■ All specified inbound services (and no others)

were available to external clients.

■ The firewall management console was not

available to any users unless authenticated.

■ The firewall was resistant to a range of known

denial-of-service (DoS) tests.

■ The firewall did not allow uncontrolled access to

either the internal or demilitarized zone (DMZ)


West Coast Labs Testing Team

All West Coast Labs tests are carried out by fully

trained content and perimeter security test

engineers under the direction of the CTO Jon

Stearn, an acknowledged technical authority

among his peers, who has over 25 years

experience in the IT and security industries.

Particular thanks go to Michael Parsons, Matt

Garrad, Richard Thomas and Mike McMenamin.



Introduction continued...

■ The underlying operating system was hardened

and not vulnerable to known OS-specific attacks.

Tests were repeated as follows:

■ Probe the internal network from the Internet.

■ Probe the DMZ from the Internet.

■ Probe the firewall from the Internet.

■ Probe the external network from the internal

network (test security policy).

■ Probe the DMZ from the internal network.

■ Probe the firewall from the internal network.

Management of the firewall was evaluated using the

following criteria:

■ The local console must be secure.

■ The management console should not be open to

the external network.

■ The firewall configuration should be fully protected

and tamper-proof (except from an authorized

management station).

■ Authentication should be required for local


■ Authentication and an encrypted link should be

available for remote administration.

■ All attacks should be logged with date and time.

AV detection functionality

The testing reported on the following virus detection


■ Products were tested in accordance with

Checkmark AV Level 1 to determine their ability to

detect viruses. (West Coast Labs uses live viruses.

It does not use simulators.)

■ Multi-part viruses were reproduced in their various

manifestations and must be detected in each place

in which an infection may occur.

■ Polymorphic viruses were replicated to a minimum

(usually) of 250 iterations and must be detected in

each iteration.

Performance Tests

The following performance tests were carried out on the

firewall technology, details of the results can be found in

the White Papers online.

■ Throughput measured the maximum transmission

rate at which the firewall can forward IP traffic

without frame loss.

■ Frame loss measured the percentage of frames

lost from flows and groups sent through the firewall

that should have been forwarded.

■ Tests also determined under what load (number of

packets per second and size of packets) the

firewall began to drop packets.

■ Latency calculated the minimum, maximum, and


average latency of received frames in flows and

groups of flows sent through the firewall.

■ Maximum connection rate measured the maximum

rate of connection requests that the product could

service without dropping connections.

■ In addition a range of tests were run to evaluate

overall performance with a typical mix of

background traffic (HTTP/SMTP/FTP) consistent

with the deployment profile. Testing aimed to

assess the appliance’s ability to perform under

significant load: syn flood, udp flood, and other

malicious attacks.

■ Tests were also be carried out to assess the

product’s ability to continue performing under

sustained worm/virus attack with multiple

simultaneous attacks on the external interface.

Both the firewall and the AV performance were

taken into consideration during these tests.

■ It was expected that all attacks would be blocked

and recorded.

Find the full results online

The analysis and full test results for each solution, which include

both functionality and performance data, are online at along with white papers, buyer’s guides

and other product information.


Juniper Networks NetScreen-5GT

DEVELOPER’S STATEMENT: For IT managers who need an

advanced security appliance with superior price/performance and

manageability to protect against all manner of network attacks. Ideal for

remote offices, retail outlets and fixed telecommuters.

Product: Juniper Networks NetScreen-5GT

Manufacturer: Juniper Networks

Contact details:

Full white paper:

The NetScreen-

5GT from Juniper

Networks is


certified to Anti

Virus Level 1 and

Firewall Level 1

The Juniper Networks NetScreen-5GT

incorporates a stateful firewall with deep

inspection, VPN, anti-virus and web filtering

capabilities in one enterprise-class, all-in-one


Management can be carried out at the command

line,(locally or across the network), or using the

attractive and intuitive web interface.

The WUI allows for quick navigation and detailed

control over the entire operation of the appliance. By

default, access is restricted to HTTPS on the internal

network, but, like most other features of the appliance,

this can be reconfigured by the administrator as


The main menu in the web interface consists of clearly

marked sections and subsections: options are easy to

find and well grouped.

Testing of the firewall technology was conducted

within the framework of the Checkmark Firewall Level 1

certification test criteria.

The default configuration of the NetScreen-5GT

allows common outbound services : Telnet, FTP, HTTP,

SSL/HTTPS, SMTP and DNS. All inbound traffic is


Probes of both the internal network and the appliance

from the internet confirmed no ports open, as expected.

All attempts to pass traffic to prohibited ports failed.

Probes conducted from the internal network confirmed

that the full specified range of services were available.

There were no open ports on the appliance other than

those used for management. During a denial-of-service

attack, the NetScreen-5GT continued to allow normal

traffic flow whilst repelling all attempted break ins.

The initial configuration does not include a

demilitarized zone (DMZ) set up, but it is possible to

change the organization of the five network ports on the

rear of the device to provide several different

configuration options, including DMZ functionality if


In order to test the DMZ functionality, the NetScreen

5GT was reconfigured so that the test machines in the

DMZ had full outbound access to the internet and the

internal networks, and had access to the Telnet and

HTTP/HTTPS ports on the firewall.

Probes launched from the DMZ confirmed that

designated services were available on both the internal

and external networks, and on the NetScreen-5GT.

Attacks launched from both the internal and external

networks against hosts on the DMZ were unable to

access any services other than those specified in the

firewall policy.

Probes from the external connection to both the

internal connection and the device itself, showed that

no inbound services had been allowed through by the

reconfiguration. Probes from the internal network also

showed that there had been no change in the outbound

services allowed.

Throughout all reconfigurations employed, the

NetScreen-5GT performed exactly in accordance with

the behaviour set down in the firewall rulebase.

Logging is thorough, and configurable. Logs may be

viewed in the web interface and exported in plain text

for analysis. All relevant information was correctly

logged throughout the course of testing.

AV functionality is provided by a Trend Micro engine

and provides scanning of HTTP, FTP, POP3 and SMTP

traffic. There is a wide range of configuration options


In testing over a number of months against the

Checkmark AV Level 1 certification criteria the product

achieved a 100 percent detection rate.

The NetScreen-5GT has the smallest footprint of the

devices on test, but as the results of the firewall and AV

detection results show, its size belies the ease of use of

the interface and the ability of this appliance toperform

well under pressure.


Juniper Networks NetScreen-5GT is a fully

featured security appliance with a small footprint.

The configurable firewall provides excellent

performance. The proven anti virus screening

covers a wide range of network protocols. It

should be considered by any

administrator who wants full

control over perimeter security.



Equiinet NetPilot Plus

DEVELOPER’S STATEMENT: Equiinet specialises in the manufacture

of multi-functional smart unified threat management appliances that

provide secure Internet access for small and medium sized enterprises.

Equiinet has over 30,000 of its products installed in the U.K.

Product: Equiinet NetPilot Plus

Manufacturer: Equiinet

Contact details:

Full white paper:

The NetPilot Plus

from Equiinet is


certified to Anti

Virus Level 1,

Firewall Level 1

and VPN

The NetPilot Plus has been designed and

developed to provide an all-in-one solution to the

problems of network security. It offers a range of

unified threat management features including firewall,

antivirus, anti-spam, and intrusion detection/prevention

capabilities, together with VPN capabilities. For the

purposes of this AV firewall report, only the virus

detection and firewall technologies were tested.

The appliance can be managed via a text interface

using an attached monitor and keyboard, or by the

simple but attractive web interface

User options on the web interface are well grouped

and clearly labelled. They provide quick configuration

for the normal day-to-day operations of a small to

medium sized office. The presentation of the traffic data

in a graphical form helps to highlight any intrusion

attempts and adds to the overall ease of use, especially

for those users who find it difficult to interpret network

activity from purely text based logs.

Prior to testing of the firewall, the NetPilot Plus was

reset to factory defaults. The IP addresses of the

internal and external interfaces were set at the text

interface using an attached keyboard, and all further

configuration performed using the web interface.

The default rules allow only HTTP and SSL/HTTPS

traffic as outbound services and only inbound SMTP

traffic, directed to the device itself, which hosts an

easily configurable mail server. No inbound traffic was

allowed through to the internal network.

Probes of the internal network from the internet

revealed nothing, whilst a probe of the appliance itself

confirmed the only allowed inbound service to be


All attempts to pass traffic to non-allowed services,

including DNS, failed.

Probes from the internal network showed that only

HTTP and SSL/HTTPS traffic was allowed to pass to

external hosts, and attempts to connect to other

services failed.

During a directed denial-of-service attack the NetPilot

Plus allowed legitimate traffic out from the internal

network whilst stopping any attempted incursions from

violating the firewall.

After reconfiguration of the appliance to provide a

wider range of outbound services, probes from the

internal network against the external connection

showed that the full range of services were available, as

was to be expected. Conversely, only those inbound

services that had been specifically allowed on the

NetPilot Plus were accessible from the internet. Any

attempt to access prohibited services on the internal

network was still not allowed.

Even when under extended attack the NetPilot Plus

logged all attacks and dropped packets with time and

date as well as other relevant data, including source

and destination IP addresses and ports, and MAC

addresses. The web management interface also

provides a useful visual graphical representation, which

gives an indication of network traffic and can act as a

warning light to point an administrator to look at the

detailed text logs.

NetPilot Plus successfully achieved the standard

required for Checkmark the Firewall Level1


An additional license must be purchased to enable the

AntiVirus functionality of the NetPilot Plus The

appliance’s Sophos engine provides scanning of web

and email traffic. In tests, the NetPilot Plus successfully

detected 100 percent of the viruses in the May 2005

virus collection over both HTTP and SMTP protocols.

As the results of the firewall and AV detection results

show, the NetPilot Plus appliance is straightforward to

set up, configure and use and, as the performance test

data in the online White Paper shows, it is an effective

and efficient appliance.


The Equiinet NetPilot Plus has a simple

interface for a complex security device. Default

settings are well chosen and allow quick and

easy deployment, with the potential for more

complex configuration. Proven AV protection for

HTTP and SMTP is included.



SecurePipe Managed Network Security

DEVELOPER’S STATEMENT: SecurePipe delivers managed network

security services to organizations impacted by regulatory requirements.

Via 24x7x365 monitoring and management and dynamic reporting, it

helps clients strengthen security, reduce costs and improve compliance.

Product: SecurePipe Managed Network


Company: SecurePipe

Contact details:

Full white paper:

The SecurePipe

Managed Network


technology is


certified to Anti

Virus Level 1,VPN

Firewall Level 1

and Firewall

Level 2.

The approach here is somewhat different to the

other products on test, as SecurePipe provides a

managed service. Customers specify exactly

what services they require and SecurePipe configures

and maintains the firewall accordingly.

All requests for changes to the firewall configuration

are made via SecurePipe’s Security Console website

and are implemented by their technical team. The

website is secured with SSL and access is restricted by

username and password.

The initial set up requested for the SecurePipe

solution included no inbound servicesand restricted

outbound services. The appliance was shipped ready to

plug in to the test network with this configuration

already set up.

Probes of both the internal network and the device

itself from the external network revealed nothing and all

attempts to pass prohibited traffic or bypass the

restrictions failed.

Throughout a directed denial-of-service (DoS) attack

the SecurePipe solution allowed legitimate traffic out

from the internal network while stopping any rogue

packets from getting through to the internal network.

To test the demilitarized zone (DMZ) functionality the

SecurePipe box was reconfigured by technical support

to allow a DMZ on one of the ports via an expansion

card. The configuration allowed access to specific ports

on specific hosts on the DMZ: HTTP on one, FTP on

another, and so on. Probes confirmed that any attempts

to access other services were completely blocked,

exactly as requested. The internal network remained

completely protected, and outbound services were


The box was then reconfigured again to disable

access to some outbound services. The availability of

those services was confirmed by test from the external

network; however any attempt to connect to those

services from the internal network was met with an

error message saying that the connection had been

blocked at the proxy level.

This gives the rather interesting option of being able to

see that services exist on remote servers while being

unable to connect to them.

The SecurePipe web interface can provide detailed

text logs of dropped and blocked traffic along with some

system logs. These packet logs provide date and time

as well as other relevant data. Information is also

provided about network attacks and Securepipe will

also send emails to administrators of domains which

have been the source of attacks: these too can be

viewed online.

SecurePipe can provide a managed anti-virus solution

for SMTP. This requires amendment of the MX records

for the domain to be protected to pass all mail through

the Securepipe service. Functionality was tested

against the requirements of the Checkmark certification

for AV Level 1.

When tested against the May 2005 virus collections

the SecurePipe service intercepted all viruses sent by

email to the target domain.

The SecurePipe managed service does not remove

responsibility for a secure network from the local

administrator. It allows the administrator to specify what

security policy they wish to have in place, but leaves the

implementation and monitoring of that policy to security


The online support request form was not used during

testing of the service, but the technical support proved

to be knowledgeable and effective.


The SecurePipe managed service allows an

administrator to specify their security policy

and leave the implementation of that policy to

external security professionals. AV support is

available for SMTP. An excellent solution for any

hard pressed IT department.


In the dark

when it comes to

choosing the right

Anti Virus, Trojan,

Spyware, Firewall

and VPN solution?

Check for the Checkmark

The Checkmark System independently tests and certifies that security products genuinely

achieve internationally recognised standards. West Coast Labs’ independent testing

laboratories have a worldwide reputation for accuracy and reliability. The Checkmark Systems

tests products regularly, in some cases as frequently as every six weeks, to ensure that the

product maintains compliance with the international standards.

If the product your using doesn’t have one, maybe you should ask why.

To find out more about the Checkmark visit our website at

The following companies have products tested and certified under the Checkmark system:

AhnLab • Aladdin • Alcatel • Blackspider • Cat • Command • Computer Associates • Cybersoft

• Equiinet • ESET • F-Secure • GData • GFI • Grisoft • Hauri • ISS • Juniper Networks •

Kaspersky • McAfee • Microworld • NGS Software • Norman • Panda • Preventon • Rapid 7 •

SecurePipe • Softwin • Sophos • Symantec • Trend Micro • VirusBuster • Wanadoo

More magazines by this user
Similar magazines