27.10.2013 Views

The BreakingPoint Resiliency Score - Fortinet

The BreakingPoint Resiliency Score - Fortinet

The BreakingPoint Resiliency Score - Fortinet

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

FortiGate-3950B <strong>Score</strong>s 95/100 on <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

(Security, Performance, & Stability)<br />

Overview<br />

<strong>Fortinet</strong>® FortiGate®-3950B enterprise consolidated security appliance has<br />

achieved a <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> of 95/100 - the highest<br />

published score on record. <strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> is a test based<br />

on industry standards for performance, security and stability of network and<br />

security devices. <strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> is presented as a<br />

numeric grade from 1 to 100. Networks and devices may receive no score if<br />

they fail to pass traffic at any point or they degrade to an unacceptable<br />

performance level.<br />

<strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> establishes standards against which<br />

network and security devices are measured. It provides automated,<br />

standardized and deterministic methods for evaluating and ensuring the<br />

resiliency of networks, network equipment and data centers. <strong>The</strong><br />

<strong>BreakingPoint</strong> Cyber Tomography Machines (CTM) provides a standard<br />

measurement of the performance, security and stability of networks and data<br />

centers using real-world application traffic, real-time security attacks, extreme<br />

user load and application fuzzing.<br />

<strong>The</strong> FortiGate-3950B appliance, running FortiOS 4.0 MR3 and utilizing<br />

its two built-in 10-GbE interfaces was tested. <strong>The</strong> evaluation was<br />

performed using one <strong>BreakingPoint</strong> Storm CTM containing one four-port<br />

10-GbE card.<br />

<strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> validates the performance of <strong>Fortinet</strong>’s network security solutions using metric-based,<br />

rigorous real-world testing. <strong>Fortinet</strong> customers can deploy products with the utmost confidence, as these standards-based<br />

evaluations enable enterprises to easily determine the right fit for their environments. As demonstrated by the <strong>BreakingPoint</strong><br />

<strong>Resiliency</strong> <strong>Score</strong> testing, the FortiGate-3950B appliance continues to impress with its security capabilities, scalability, flexibility<br />

and stability.<br />

For more information on the <strong>Fortinet</strong> FortiGate-3950B series please visit<br />

http://www.fortinet.com/products/fortigate/3950series.html<br />

FortiGate-3950B: Modular Security Platform<br />

Download the <strong>BreakingPoint</strong> whitepaper, “A Six-Step Plan for Competitive Device Evaluations” here<br />

http://www.breakingpointsystems.com/resources/white-papers/six-step-plan-competitive-device-evaluation-bakeoffs/


Firewall <strong>Resiliency</strong> Report<br />

1. Firewall Report<br />

Product: <strong>Fortinet</strong> FortiGate-3950B<br />

<strong>Resiliency</strong> <strong>Score</strong>: 95<br />

2. Synopsis<br />

Throughput 64: 93.75<br />

1518: 98.43<br />

Sessions Count: 100.00<br />

Rate: 79.94<br />

Rate: 100.00<br />

Robustness IP: pass<br />

UDP: pass<br />

TCP: pass<br />

Lab Real Session Stress Session Rate Stress<br />

Rate: 93.28<br />

Count: 100.00<br />

Security pass pass pass pass<br />

Overall <strong>Score</strong> 95.056404<br />

Throughput<br />

<br />

Sessions<br />

Measures a device’s ability to handle large numbers of TCP sessions and the rate at which it handles them.<br />

Robustness<br />

<br />

Security<br />

<br />

Overall <strong>Score</strong><br />

A blended average of all sub-test. This number represents the over score relative to<br />

expected performance for the resiliency test.


2.1. <strong>Score</strong> Calculation<br />

Overall <strong>Score</strong><br />

Calculation:(A(93) + B(98) + C(93) + D(100) + E(100) + F(79) + G(100)) / 7<br />

Overall <strong>Score</strong> = 95.06<br />

Throughput<br />

A) I EEE Throughput measurement of 64 byte frames A = (100 X Throughput Achieved[w/64]) / Max Wireline<br />

Throughput = (100 X 9375) / 10000 <strong>Score</strong> = 93<br />

B) IEEE Throughput measurement of 1518 byte frames B = (100 X Throughput Achieved[w/1518]) / Max Wireline<br />

Throughput = (100 X 9843) / 10000 <strong>Score</strong> = 98<br />

C) Throughput, Simulated Real World Conditions C = (100 X Application Frames Rate) / Max Application Frames<br />

Rate = (100 X 1399140) / 1500000 <strong>Score</strong> = 93<br />

Sessions<br />

D) Concurrent IETF 793 TCP Connections D = (100 X Number Flows) / Max Number Flows = (100 X 10000000) /<br />

10000000 <strong>Score</strong> = 100<br />

E) Concurrent IETF 2581 TCP and IETF 768 UDP Connections E =(100 X Number Flows) / Max Number Flows =<br />

(100 X 10069955) / 10000000 <strong>Score</strong> = 100<br />

F) IETF 793 TCP Connections/sec F = (100 X Flow Rate / Max Flow Rate = (100 X 119908) / 150000) <strong>Score</strong> = 79<br />

G) IETF 2581 TCP and IETF 768 UDP Connections/sec G = (100 X Flow Rate) / Max Flow Rate = (100 X 121114)<br />

/ 100000 <strong>Score</strong> = 100<br />

Robustness<br />

H) IETF 7 91 IP Stack Stability H = Dropped Pings <strong>Score</strong> = pass<br />

I) IETF 768 UDP Stack Stability I = Dropped Pings <strong>Score</strong> = pass<br />

J) IETF 79 3 TCP Stack Stability J = Dropped Pings <strong>Score</strong> = pass<br />

Security<br />

K) CVE Security F ault Injection, Independent K = Dropped Pings <strong>Score</strong> = pass<br />

L) CVE Security Fault Injection, Benign L = Dropped Pings <strong>Score</strong> = pass<br />

M) CVE Security Fault Injection, Concurrent Sessions Stress M = Dropped Pings <strong>Score</strong> = pass<br />

N) CVE Security Fault Injection, Session Rate Stress N = Dropped Pings <strong>Score</strong> = pass


2.2. Throughput<br />

<br />

for the device.<br />

Network Packet Stress<br />

This test will be repeated once with 64 byte and again with 1518 byte packets. <strong>The</strong>se two packet sizes represent the<br />

smallest and larget valid packet size for a single network frame. <strong>The</strong> test begins by transmitting packets at half of<br />

the theoretical maximum rate for the given packet size. Any dropped or corrupted packets result in a failed iteration.<br />

Testing continues iterating in a binary search pattern;; for each iteration testing at a rate halfway between the last<br />

passed test and the last failed test, until a maximum successful rate is found.<br />

Benign Realistic Network Packets<br />

<br />

bandwidth. <strong>The</strong> test begins by transmitting packets at half of the theoretical maximum rate for the given packet size.<br />

Any dropped or corrupted packets result in a failed iteration. Testing continues iterating in a binary search pattern;;<br />

for each iteration testing at a rate halfway between the last passed test and the last failed test, until a maximum<br />

successful rate is found.<br />

2.3. Sessions<br />

<br />

TCP Sessions Stress<br />

<br />

<br />

test begins by opening a single TCP session. Every 5 seconds, 1500 additional TCP sessions are attempted, up to a<br />

maximum of 10,000,000 concurrent TCP sessions. Once completed, an analysis is made to determine the achieved<br />

concurrent sessions based on the measured number of active concurrent TCP sessions.<br />

Benign Realistic Network Sessions<br />

This test begins by opening a single TCP session, which remains open for the duration of the test. Every 5 seconds<br />

1500 additional TCP sessions are attempted, up to a maximum of 10,000,000 concurrent TCP sessions. Once<br />

completed, an analysis is made to determine the achieved concurrent sessions based on the measured number of<br />

active concurrent TCP sessions.


2.4. Robustness<br />

<br />

IP Robustness<br />

<br />

<br />

Packets will have random payload ranging in size from 46 to 1500 bytes, and will be transmitted at a rate between<br />

2000 and 2500 packets per second. Data will be transmitted for at least one hour, for a minimum of 5,000,000 distinct<br />

<br />

the Urgent pointer, and the IP checksum.<br />

UDP Robustness<br />

<br />

<br />

of the UDP header in addition to the IP header. Packets will have random payload ranging in size from 46 to 1500<br />

bytes, and will be transmitted at a rate between 2000 and 2500 packets per second. Data will be transmitted for at<br />

<br />

TCP Robustness<br />

This test limits the scope of random testing to Layer 4 by randomizing portions of the TCP header in addition to the<br />

IP header. Packets will have random payload ranging in size from 46 to 1500 bytes, and will be transmitted at a<br />

rate between 2000 and 2500 packets per second. Data will be transmitted for at least one hour, for a minimum of<br />

<br />

2.5. Security<br />

<br />

Security - Laboratory conditions<br />

<br />

payloads are blocked or neutered.<br />

Security - Benign Realistic Conditions<br />

<br />

<br />

<br />

Security - Concurrent Sessions Stress Conditions<br />

<br />

<br />

<br />

as measured in the performance baseline.<br />

Security - Session Open Rate Stress Conditions<br />

<br />

<br />

<br />

performance baseline.


2.6. Settings<br />

Setting Value<br />

Speed 10.00 Gigabits<br />

Device Type Firewall<br />

Run Type Full<br />

Session Rate yes<br />

Robustness yes<br />

Throughput yes<br />

Security yes<br />

<br />

Client Routing Server Routing <br />

Network:192.168.50.0/<br />

24 Min:192.168.50.3<br />

Max:192.168.50.254<br />

DUT Address:192.168.50.1<br />

Network:192.168.50.0/ 24<br />

Gateway:192.168.50.2<br />

- End of Document -<br />

DUT Address:192.168.51.1<br />

Network:192.168.51.0/ 24<br />

Gateway:192.168.51.2<br />

Network:10.0.0.0/8 Min:10.0.0.1<br />

Max:10.255.255.254


<strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

An automated, standardized, and deterministic measure of the performance, security, and stability of network<br />

and application infrastructure devices and systems<br />

1


Table of Contents<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

Introduction .......................................................................................................................................................................3<br />

<strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> ................................................................................................................................................................ 3<br />

<strong>Resiliency</strong> <strong>Score</strong> Assessment Elements .......................................................................................................................................................... 3<br />

Prescribed Conguration .................................................................................................................................................................................... 3<br />

Duration .................................................................................................................................................................................................................... 4<br />

Results ........................................................................................................................................................................................................................ 4<br />

<strong>Resiliency</strong> <strong>Score</strong> Phases .....................................................................................................................................................4<br />

Phase 1: Throughput ............................................................................................................................................................................................. 4<br />

64-Byte Packet ................................................................................................................................................................................................. 5<br />

1518-Byte Packet Measurement ............................................................................................................................................................... 5<br />

Real Bandwidth Measurement .................................................................................................................................................................. 6<br />

Phase 2: Sessions .................................................................................................................................................................................................... 6<br />

Basic Concurrent Sessions ........................................................................................................................................................................... 6<br />

Real Concurrent Sessions ............................................................................................................................................................................. 7<br />

Basic Connections per Second................................................................................................................................................................... 8<br />

Real Connections per Second .................................................................................................................................................................... 9<br />

Phase 3: Robustness.............................................................................................................................................................................................. 9<br />

IP Stability .......................................................................................................................................................................................................... 9<br />

UDP Stability ..................................................................................................................................................................................................... 9<br />

TCP Stability ...................................................................................................................................................................................................... 9<br />

Phase 4: Security .................................................................................................................................................................................................... 10<br />

Independent Security ................................................................................................................................................................................... 10<br />

Benign Trac Security .................................................................................................................................................................................. 10<br />

Concurrent Sessions Security ..................................................................................................................................................................... 10<br />

Session Open Rate Security ........................................................................................................................................................................ 10<br />

2


www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

<strong>Resiliency</strong> <strong>Score</strong> Setup .......................................................................................................................................................11<br />

Physical Connection ............................................................................................................................................................................................. 11<br />

Network Conguration ........................................................................................................................................................................................ 11<br />

Switch ................................................................................................................................................................................................................. 11<br />

Router ................................................................................................................................................................................................................. 12<br />

Load Balancer ................................................................................................................................................................................................... 12<br />

Proxy .................................................................................................................................................................................................................... 13<br />

Firewall ............................................................................................................................................................................................................... 13<br />

Intrusion Prevention System ...................................................................................................................................................................... 14<br />

Unied Threat Manager ................................................................................................................................................................................ 14<br />

Summary .............................................................................................................................................................................14<br />

About <strong>BreakingPoint</strong> Storm CTM .....................................................................................................................................15<br />

3


Introduction<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

Organizations want measurable answers, not assurances, when it comes to network and application performance, security, and stability.<br />

We have come to expect evaluation and certication of product performance for everything from our phones to our automobiles—yet<br />

network and application infrastructures and the equipment upon which they rely have no standardized certication for performance and<br />

security. Instead, buyers must trust statements made in product marketing literature, which are based on best-case scenarios, not real-world<br />

evidence.<br />

<strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> puts an end to this problem by establishing a standard against which networks and security devices<br />

are measured. This document describes a step-by-step approach for determining a <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong>, as implemented in<br />

version 2.0 of the <strong>BreakingPoint</strong> Storm Cyber Tomography Machine (CTM) rmware.<br />

<strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

<strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> provides an automated, standardized, and deterministic method for evaluating and ensuring the<br />

resiliency of networks and network equipment. This feature of the <strong>BreakingPoint</strong> Storm CTM provides a standard measurement of the<br />

performance, security, and stability of every component of the network and data center using real-world application trac, real-time<br />

security attacks, extreme user load, and application fuzzing.<br />

A <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> is calculated using standards by organizations such as<br />

US–CERT, IEEE, and the IETF, as well as real-world trac mixes from the world’s largest<br />

service providers. A <strong>BreakingPoint</strong> Storm CTM user simply selects the network or device<br />

for evaluation and the speed at which the device or system is required to perform for an<br />

automated measurement of resiliency. <strong>The</strong> <strong>BreakingPoint</strong> Storm CTM then subjects the<br />

device to a battery of simulations using a blended mix of application trac and malicious<br />

attacks, including obfuscations. <strong>The</strong> <strong>BreakingPoint</strong> Storm CTM delivers this measurement in<br />

the form of a <strong>Resiliency</strong> <strong>Score</strong>—much like an Underwriters Laboratories (UL) certication.<br />

<strong>The</strong> <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> is presented as a numeric grade from 1 to 100. Networks<br />

and devices may receive no score if they fail to pass trac at any point or degrade to an<br />

unacceptable performance level. <strong>The</strong> <strong>Resiliency</strong> <strong>Score</strong> takes the guesswork and subjectivity<br />

out of validation and allows administrators to quickly understand the degree to which system security will be impacted under load and<br />

under the threat of newly evolved cyber attacks.<br />

<strong>Resiliency</strong> <strong>Score</strong> Assessment Elements<br />

<strong>The</strong>re are four phases of the <strong>Resiliency</strong> Scoring process—each with a set of associated measurements—that target specic aspects of<br />

behavior:<br />

• Throughput Phase<br />

• Sessions Phase<br />

• Robustness Phase<br />

• Security Phase<br />

Prescribed Conguration<br />

Because the <strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong> is a standardized measurement designed to maintain the consistency of scores across vendors,<br />

the <strong>BreakingPoint</strong> Storm CTM requires little or no conguration. <strong>The</strong> user simply chooses a class of device from a set list of options; that<br />

choice determines the network conguration that will be used. <strong>The</strong> user congures their own device to support that network conguration,<br />

then selects a bandwidth limit for the device, in 100 Mbps increments.<br />

4


Duration<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

<strong>The</strong> <strong>Resiliency</strong> <strong>Score</strong> process can be run in either “Full” or “Quick” mode. <strong>The</strong> “Quick” option allows the user to work through initial issues<br />

without waiting through the turnaround time required for a full resiliency assessment. However, only the “Full” option provides an actual<br />

<strong>Resiliency</strong> <strong>Score</strong>.<br />

Results<br />

Once a full assessment encompassing all elements is complete, resiliency is scored from 0 to 100 by calculating the results from individual<br />

phases of the <strong>Resiliency</strong> <strong>Score</strong> process. If the network or device fails any part of the assessment, no score is given, and the assessment is<br />

declared “Failed.”<br />

<strong>Resiliency</strong> <strong>Score</strong> Phases<br />

<strong>The</strong>re are four phases of the process for producing a <strong>Resiliency</strong> <strong>Score</strong> for a component of a network or data center:<br />

Phase 1: Throughput<br />

<strong>The</strong> throughput phase targets the device’s basic packet forwarding ability. It consists of three parts:<br />

64-Byte Packet<br />

To determine the maximum achievable bandwidth of a device when forwarding 64-byte packets only, the <strong>BreakingPoint</strong> Storm CTM<br />

executes a series of simulations in a binary search pattern. For each one, success or failure is determined based on the following criteria:<br />

• <strong>The</strong> number of received frames must equal the number of transmitted frames to ensure that there were no dropped packets.<br />

• No corrupted frames should be received.<br />

Initially, trac is generated at 50 percent of the user-specied rate of the device. If the device passes this assessment, the next rate attempt<br />

is halfway between the initial rate and the maximum rate, or 75 percent of the user-specied rate. If the device fails, the next rate attempted<br />

is 25 percent. <strong>The</strong> process continues for seven iterations to determine the maximum achieved rate within a 1 percent margin of error.<br />

<strong>The</strong> individual score for this portion of the overall <strong>Resiliency</strong> <strong>Score</strong> is based on the measured bandwidth of the device and is expressed as<br />

the percentage of the measured speed versus the user-congured speed. <strong>The</strong> results of the 64-byte packet assessment are shown in the<br />

following way in section 2.1 of the report:<br />

A) IEEE Throughput measurement of 64 byte frames<br />

A = max(100,100 * Speed(999)<br />

/ MaxSpeed(1000))<br />

=100.00<br />

1518-Byte Packet Measurement<br />

Next, a second binary search is run with the frame size set to 1518. Aside from the frame size, this component runs identically to the 64-byte<br />

packet assessment and is scored the same way. <strong>The</strong> results below are provided in the nal report:<br />

B) IEEE Throughput measurement of 1518 byte frames<br />

B = max(100,100 * Speed(999)<br />

/ MaxSpeed(1000))<br />

=100.00<br />

5


Real Bandwidth Measurement<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

For the nal part of the Throughput Phase, the device’s throughput is measured when presented with a realistic blend of application<br />

trac. <strong>The</strong> <strong>BreakingPoint</strong> Application Simulator transmits at a bandwidth no higher than the user-congured maximum, with maximum<br />

concurrent sessions set to 10,000,000, and a load prole set to control the session open rate.<br />

For a full <strong>Resiliency</strong> <strong>Score</strong>, the load prole begins by opening sessions at a rate of 100 sessions per second. Every 5 seconds, the attempted<br />

open rate is increased by another 100 sessions per second, until 750,000 sessions per second is reached. <strong>The</strong> quick version of the assessment<br />

proceeds in a similar way, except the initial rate is 10,000, and the attempt rate is increased by 10,000 every ve seconds.<br />

<strong>The</strong> bandwidth achieved is determined by searching for the maximum measured received frame rate that was sustained for at least three<br />

seconds. This bandwidth achieved is factored into the score as a percentage of the congured bandwidth and is also used as a basis to<br />

determine the rate of some subsequent processes, as detailed below.<br />

C) Throughput, Real<br />

C= max(100,100 * Application Frame Rate(1420140)<br />

/ Max Application Frame Rate(150000))<br />

=100.00<br />

Phase 2: Sessions<br />

<strong>The</strong> Sessions Phase of the <strong>Resiliency</strong> <strong>Score</strong> focuses on the device’s ability to support UDP and TCP sessions, in terms of both rate and<br />

number of concurrent sessions.<br />

Basic Concurrent Sessions<br />

During the Sessions Phase, the ability of the device to process straightforward TCP sessions is determined using <strong>BreakingPoint</strong> Session<br />

Sender. <strong>The</strong> bandwidth limit is set to the user-congured speed for the device. <strong>The</strong> steady-state behavior for the TCP sessions is set to “Hold,”<br />

so that sessions stay open for the duration of the assessment.<br />

A load prole is generated to control the maximum number of concurrent sessions. For a full <strong>Resiliency</strong> <strong>Score</strong>, 5,000 sessions are opened<br />

at the beginning. Every ve seconds, 5,000 more sessions are opened, up to a maximum of 10,000,000 concurrent sessions. Once the<br />

6


www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

maximum is reached, the sessions are held open for an additional 30 seconds before wrapping up the assessment. A quick <strong>Resiliency</strong> <strong>Score</strong><br />

follows similar behavior, except that it starts with 500,000 sessions and adds another 500,000 every ve seconds.<br />

For scoring, the maximum achieved number of concurrent sessions that was sustained for at least three seconds becomes the measured<br />

maximum. This is applied to the score as a percentage of the 10-million-session maximum.<br />

D) Concurrent IETF 793 TCP Connections<br />

D = max(100,100 * Number Flows(10000000)<br />

/ Max Number Flows(10000000))<br />

=100.00<br />

Real Concurrent Sessions<br />

<strong>The</strong> Sessions Phase also measures the number of concurrent TCP sessions supported by the device using the <strong>BreakingPoint</strong> Application<br />

Simulator and the <strong>BreakingPoint</strong> Enterprise application prole. <strong>The</strong> bandwidth limit is set to the user-congured speed for the device.<br />

Again, the steady-state behavior for the TCP sessions is “Hold,” so that they stay open for the duration. <strong>The</strong> operation is identical to the<br />

<strong>BreakingPoint</strong> Session Sender assessment.<br />

E) Concurrent IETF 2581 TCP and IETF 768 UDP Connections<br />

E = max(100,100 * Number Flows(10066931)<br />

/ Max Number Flows(10000000))<br />

=100.00<br />

7


Basic Connections per Second<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

<strong>The</strong> basic connections per second measurement uses <strong>BreakingPoint</strong> Session Sender to generate the simple TCP connections required to<br />

measure the maximum rate at which the device can open new TCP connections. <strong>The</strong> bandwidth limit for <strong>BreakingPoint</strong> Session Sender is set<br />

to the user-specied rate for the device, and the maximum number of cumulative sessions is set to 10,000,000.<br />

When performing a full <strong>Resiliency</strong> <strong>Score</strong>, this process begins by opening TCP sessions at a rate of 100 per second. Every ve seconds, the<br />

attempt rate is increased by another 100 sessions per second. This continues until the attempted rate is 750,000 connections per second.<br />

Once the maximum is reached, the process continues at that rate for 30 more seconds.<br />

<strong>The</strong> “Quick” resiliency assessment is conducted in a similar way, except that the rate starts at 10,000 and is increased by another 10,000<br />

every ve seconds.<br />

For scoring purposes, the <strong>BreakingPoint</strong> Storm CTM evaluates the maximum achieved connection rate that was held for at least three<br />

seconds. <strong>The</strong>n a rate goal is calculated as follows:<br />

• For a user-specied bandwidth of more than 1Gbps, the goal is 150,000.<br />

• For user-specied bandwidths of 1Gbps or less, the goal is the bandwidth in Gbps times 150,000. For example, for a user-specied<br />

bandwidth of 500Mbps, the goal would be half of 150,000, or 75,000.<br />

<strong>The</strong> maximum measured rate is then incorporated into the score as a percentage of the determined goal:<br />

F) IETF 793 TCP Connections/sec<br />

F = max(100,100 * Number Flows(181390)<br />

/ Max Number Flows(150000))<br />

=100.00<br />

8


Real Connections per Second<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

<strong>The</strong> data from the throughput “Real Bandwidth” measurement is reused to evaluate real connections per second. <strong>The</strong> same process is run<br />

when either throughput or sessions is measured; when both are measured, the process is run only once, and the data is used for both.<br />

<strong>The</strong> connection rate is scored in a very similar manner to the basic session rate assessment. For scoring purposes, the maximum achieved<br />

connection rate that was held for at least three seconds is determined. <strong>The</strong>n a rate goal is calculated as follows:<br />

• For a user-specied bandwidth of more than 1Gbps, the goal is 100,000.<br />

• For user-specied bandwidths of 1Gbps or less, the goal is the bandwidth in Gbps times 100,000. In other words, for a user-specied<br />

bandwidth of 500Mbps, the goal would be half of 100,000, or 50,000.<br />

<strong>The</strong> maximum measured rate is then incorporated into the score as a percentage of the determined goal.<br />

G) IETF 2581 TCP and IETF 768 UDP Connections/sec<br />

G = max(100,100 * Number Flows(120213)<br />

/ Max Number Flows(100000))<br />

=100.00<br />

Phase 3: Robustness<br />

Three processes contribute to the robustness score. Each of them is structured very similarly, using a <strong>BreakingPoint</strong> Stack Scrambler to<br />

target a dierent layer of the network stack.<br />

IP Stability<br />

To evaluate IP stability, <strong>BreakingPoint</strong> Stack Scrambler is congured to send corruptions at 12 Mbps, including bad IP version elds, bad IP<br />

options, bad “urgent” pointers, and bad checksums. A diagnostic ICMP ping request is sent every second. <strong>The</strong> ICMP packet is structured to<br />

distinguish it from any other packets, and a count is kept of how many of these are received on the other side.<br />

After this process is complete, the count of received pings is compared to the count of transmitted pings. Up to two pings not received are<br />

allowed with no penalty. A third missed ping results in one point being subtracted from the overall score. Four pings dropped result in a<br />

10-points deduction, and ve or more dropped pings result in a failing <strong>Resiliency</strong> <strong>Score</strong>.<br />

H) IETF 791 IP Stack Stability<br />

H = Dropped Pings<br />

= pass<br />

UDP Stability<br />

<strong>The</strong> IP stability process is followed by an identical UDP stability evaluation. For this process, however, the target stack is set to UDP.<br />

I) IETF 768 UDP Stack Stability<br />

I = Dropped Pings<br />

= pass<br />

TCP Stability<br />

For the nal robustness assessment, the same process is used to target the TCP stack. This assessment is also scored the same way.<br />

J) IETF 793 TCP Stack Stability<br />

J = Dropped Pings<br />

= pass<br />

9


Phase 4: Security<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

<strong>The</strong> Security Phase measures the device’s response to attack using the <strong>BreakingPoint</strong> Strike Set repeatedly under dierent network<br />

background conditions. For a full <strong>Resiliency</strong> <strong>Score</strong>, the strike set “<strong>Resiliency</strong>—All CVE Strikes” is used. This Strike Set includes all Strikes that<br />

are documented as exploits in the CVE database. A “Quick” resiliency assessment uses the Strike set “Backdoor Strikes.”<br />

Each Security Phase assessment includes a <strong>BreakingPoint</strong> Stack Scrambler component targeting the TCP layer. This produces diagnostic<br />

pings used to measure security capabilities in the same way that robustness is scored: Up to two pings not received are allowed with no<br />

penalty; a third missed ping subtracts one point from the overall score; four pings dropped result in 10 points o; and ve or more dropped<br />

lead to a failing <strong>Resiliency</strong> <strong>Score</strong>.<br />

Independent Security<br />

This measurement is initially conducted using only the <strong>BreakingPoint</strong> Security component with the <strong>BreakingPoint</strong> Stack Scrambler in active<br />

mode; in other words, there is no background trac.<br />

K) CVE Security Fault Injection, Independent<br />

K = Dropped Pings<br />

= pass<br />

Benign Trac Security<br />

Next, the same process is executed with the addition of <strong>BreakingPoint</strong> Application Simulator and the <strong>BreakingPoint</strong> Enterprise application<br />

prole. Maximum attempted sessions per second, maximum number of concurrent sessions, and throughput are all congured to a value<br />

that is half of the maximum measured in the “Real Bandwidth” / “Real Sessions per Second” process.<br />

L) CVE Security Fault Injection, Benign<br />

L = Dropped Pings<br />

= pass<br />

Concurrent Sessions Security<br />

<strong>The</strong> previous process is run again, but this time with <strong>BreakingPoint</strong> Security and Stack Scrambler components, as well as <strong>BreakingPoint</strong><br />

Application Simulator with the maximum attempted sessions per second and the throughput both congured to a value half of the<br />

maximum measured in the “Real Bandwidth” / “Real Sessions per Second” process. <strong>The</strong> number of concurrent sessions for Application<br />

Simulator is set to 10,000,000.<br />

M) CVE Security Fault Injection, Concurrent Sessions Stress<br />

M = Dropped Pings<br />

= pass<br />

Session Open Rate Security<br />

Finally, the process is executed once more with <strong>BreakingPoint</strong> Security and Stack Scrambler components, as well as <strong>BreakingPoint</strong><br />

Application Simulator with the maximum number of concurrent sessions and the throughput both congured to a value half of the<br />

maximum measured in the “Real Bandwidth” / “Real Sessions per Second” process. <strong>The</strong> maximum session open rate is congured to 750,000<br />

sessions per second.<br />

N) CVE Security Fault Injection, Session Rate Stress<br />

N = Dropped Pings<br />

= pass<br />

10


<strong>Resiliency</strong> <strong>Score</strong> Setup<br />

Physical Connection<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

<strong>The</strong> device under evaluation should be connected via two of its interfaces to the <strong>BreakingPoint</strong> Storm CTM. <strong>The</strong>se connections will be<br />

referred to as Physical Interface 1 and Physical Interface 2, based on the interface reservation on the <strong>BreakingPoint</strong> Storm CTM.<br />

Network Conguration<br />

Figure 1 - Physical Connection to the <strong>BreakingPoint</strong> Storm CTM<br />

In preparation for the <strong>Resiliency</strong> <strong>Score</strong> process, the device that will be evaluated must be congured to support the following network<br />

conguration.<br />

Switch<br />

<strong>The</strong> network conguration used to evaluate a device classied as a switch will be made up of two separate IP ranges within the same<br />

subnet, both directly attached to the device’s network, as illustrated below:<br />

Figure 2 - Logical Network, Switch<br />

11


Router<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

<strong>The</strong> network conguration used to evaluate a device that is classied as a Router is made up of two ranges of hosts in nonlocal networks.<br />

Each will arrive at the device via a router attached to a separate local subnet of the device. Trac should be routed through gateway IPs on<br />

the device, as follows:<br />

Load Balancer<br />

Figure 3 – Logical Network, Router<br />

<strong>The</strong> network conguration used to evaluate a device that is classied as a Load Balancer includes:<br />

• Trac that originates from a network of client addresses in a nonlocal subnet.<br />

• Client addresses that reach the device being scored via a router on a local subnet of interface 1.<br />

• Trac from the clients addressed to a single public IP address on interface 1 of the device being assessed.<br />

• A device being assessed that is expected to translate the destination address to one from a pool of addresses within a local subnet on<br />

interface 2 of the device being assessed.<br />

• Requests within the destination address that are handled by a set of hosts that are simulated as a multi-homed host. That is, the set of<br />

IP addresses all originate from a single MAC address, avoiding the possibility of overowing MAC tables on the device.<br />

Figure 4 - Logical Network, Load Balancer<br />

12


Proxy<br />

<strong>The</strong> network conguration used to evaluate a device that is classied as a Proxy includes:<br />

• Trac that originates from a network of client addresses in a local subnet.<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

• Client requests that are handled by a set of hosts simulated as a multi-homed host. <strong>The</strong> set of IP addresses will all originate from a<br />

single MAC address, avoiding the possibility of overowing MAC tables on the device.<br />

• Server hosts listening on a set of hosts on a nonlocal subnet.<br />

• Server addresses that are reachable by the device being assessed via a router on a local subnet of interface 2.<br />

• A device being scored that is expected to translate the source address to one from an unspecied pool, which must be reachable by<br />

Firewall<br />

the server hosts. <strong>The</strong> specic client addresses are learned as they are observed.<br />

Figure 5 – Logical Network, Proxy<br />

<strong>The</strong> network conguration used to measure the resiliency of a device classied as a Firewall includes:<br />

• Trac that originates from a network of client addresses in a local subnet<br />

• Client requests that are handled by a set of hosts that are simulated as a multi-homed host. <strong>The</strong> set of IP addresses all originate from a<br />

single MAC address, avoiding the possibility of overowing MAC tables on the device.<br />

• Server hosts that will be listening on a set of hosts on a nonlocal subnet.<br />

• Server addresses that are reachable by the device being assessed via a router on a local subnet of interface 2.<br />

• A device being assessed that is expected to translate the source address to one from an unspecied pool, which must be reachable by<br />

the server hosts. <strong>The</strong> specic client addresses are learned as they are observed.<br />

Figure 6 – Logical Network, Firewall<br />

13


Intrusion Prevention System<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

<strong>The</strong> network conguration used to measure the resiliency of an Intrusion Prevention System includes two separate IP ranges within the<br />

same subnet, both directly attached to the device’s network as illustrated below:<br />

Unied Threat Manager<br />

Figure 7 - Logical Network, Intrusion Prevention System<br />

<strong>The</strong> network conguration used to evaluate a device classied as a Unied Threat Manager will be made up of two ranges of hosts in<br />

nonlocal networks. Each will arrive at the device being assessed via a router attached to a separate local subnet of the device being<br />

assessed. Trac should be routed through gateway IPs on the device, as follows:<br />

Summary<br />

Figure 8 – Logical Network, Unied Threat Manager<br />

Based on insights into global network trac proles and using standards from US–CERT, IEEE, and the IETF, the <strong>BreakingPoint</strong> <strong>Resiliency</strong><br />

<strong>Score</strong> provides an easy-to-produce and defensible measurement of the resiliency of networks and network devices and servers. A<br />

<strong>Resiliency</strong> <strong>Score</strong> can be produced only with the <strong>BreakingPoint</strong> Storm CTM, as it is the only product capable of producing the blended mix<br />

of application trac, malicious attacks, user load, and application fuzzing required to understand how devices will perform in today’s global<br />

networks.<br />

14


About <strong>BreakingPoint</strong> Storm CTM<br />

<strong>BreakingPoint</strong> has pioneered the rst Cyber Tomography Machine with<br />

the introduction of the <strong>BreakingPoint</strong> Storm CTM. This innovative product<br />

enables users to see for the rst time the virtual stress fractures lurking<br />

within their cyber infrastructure through the simulation of crippling attacks,<br />

high-stress trac load, and millions of users. <strong>The</strong> <strong>BreakingPoint</strong> Storm<br />

CTM exposes previously impossible-to-detect vulnerabilities within cyber<br />

infrastructure components before they are exploited to destroy what is most<br />

important to you—your customer data, your assets, your reputation, and<br />

even national security. <strong>BreakingPoint</strong> Storm CTM is delivered in a three-slot<br />

chassis that provides the equivalent performance and simulation of hundreds<br />

of racks of servers:<br />

• 40 Gigabits per second of blended stateful application trac<br />

• 30 million concurrent TCP sessions<br />

• 1.5 million TCP sessions per second<br />

• 600,000+ complete TCP sessions per second<br />

• 80,000+ SSL sessions per second<br />

• 130+ stateful applications<br />

• 4,500+ live security strikes<br />

www.breakingpoint.com<br />

© 2005 - 2010. <strong>BreakingPoint</strong> Systems, Inc. All rights reserved. <strong>The</strong> <strong>BreakingPoint</strong> logo is a trademark of <strong>BreakingPoint</strong> Systems, Inc.<br />

All other trademarks are the property of their respective owners.<br />

Contact <strong>BreakingPoint</strong><br />

<strong>BreakingPoint</strong> <strong>Resiliency</strong> <strong>Score</strong><br />

Learn more about <strong>BreakingPoint</strong><br />

products and services by contacting a<br />

representative in your area.<br />

1.866.352.6691 U.S. Toll Free<br />

www.breakingpoint.com<br />

<strong>BreakingPoint</strong> Global Headquarters<br />

3900 North Capital of Texas Highway<br />

Austin, TX 78746<br />

email: salesinfo@breakingpoint.com<br />

tel: 512.821.6000<br />

toll-free: 866.352.6691<br />

<strong>BreakingPoint</strong> EMEA Sales Oce<br />

Paris, France<br />

email: emeasales@breakingpoint.com<br />

tel: + 33 6 08 40 43 93<br />

<strong>BreakingPoint</strong> APAC Sales Oce<br />

Suite 2901, Building #5, Wanda Plaza<br />

No. 93 Jianguo Road<br />

Chaoyang District, Beijing, 100022, China<br />

email: asiasales@breakingpoint.com<br />

tel: + 86 10 5960 3162<br />

15

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!