GigabitEthernet Testbed over Dark Fiber

GigabitEthernet Testbed over Dark Fiber
sl. dr. ing. Emil CEBUC *
prof. dr. ing. PUSZTAI Kalman
ing. Otto KREITER
ing. Florin FLORIAN
RoEduNet Cluj
Emil.Cebuc@cs.utcluj.ro
Pusztai.Kalman@cs.utcluj.ro
ottok@cluj.roedu.net
florin@cluj.roedu.net
Abstract
The paper will present the testing of a 120km
fiber optic link between RoEduNet NOC Cluj and
RoEduNet NOC Tg-Mures using long reach Gigabit
Ethernet technology.
1. Introduction
First we will present the hardware environment
used for the testing.
Second the software for traffic generation and
monitoring the test.
Third design of the tests and preliminary results
And last the conclusions drawn from the
performed tests.
2. Hardware Environment
The following test bed was used: Two Cisco
Catalyst 3550-24 Layer 3 switches interconnected
with a 120 km long dark fiber link. The interface
module is a CWDM-GBIC-1550 on both sides.
For load generation and monitoring we used an
other Gigabit interface. In some tests FastEthernet
interfaces were also involved.
The switch is driven by a 8.8 Gbps backplane so
it is almost impossible to insert a higher load to the
switch to make it drop packets by overloading the
switch fabric. Generating packets at a high rate with
the available equipment was also challenging.
Instead we introduced a routing loop and let packets
to bounce forth and back until TTL = 0. By
controlling the initial TTL of the IP packets the data
rate could be controlled. The test setup is shown in
figure 1.
.Figure 1
49

3. Monitoring Gigabit Interfaces
Monitoring the Gigabit interface.
To monitor with SNMP a highly loaded gigabit
interface in a Cisco Catalyst 3550 it is necessary to
take in consideration that they have just 32 Bit
SNMP counters. If the interface is fully loaded with
1 Gigabit/s traffic the counter resets every 34
seconds.
(2^32 * 8) / 10 9 = 34.35 second
In consequence it is necessary to poll the
interface at least every 30 seconds. Taking in
consideration that when the switch is heavily loaded
the switch can't respond to the SNMP request in
time we have decided to poll the interface every 20
seconds.
The monitoring machine was a RedHat Linux
2.4.19, using UCD-SNMP version 4.2.4 to fetch the
data, and RRD-tool 1.0.40 to store and graph the bit
rates and packet rates. Because in CRONTAB we
can't setup to run the data fetching script every 20
seconds we used the following script:
#!/bin/bash
while true; do
/var/local/flows/giga-test/scripts/snmp-datastore
sleep 20
done
The snmp-data-store script pools the gigabit
interface and store the data in rrd's.
IN=$(snmpget 217.73.171.3 public 2.2.1.10.26 |
awk "{print \$4 ;}")
OUT=$(snmpget 217.73.171.3 public
2.2.1.16.26 | awk "{print \$4 ;}")
INpk=$(snmpget 217.73.171.3 public
interfaces.ifTable.ifEntry.ifInUcastPkts.26 | awk
"{print \$4 ;}")
OUTpk=$(snmpget 217.73.171.3 public
interfaces.ifTable.ifEntry.ifOutUcastPkts.26 |
awk "{print \$4 ;}")
DATA=`date +%s`
rrdtool update /var/local/flows/gigatest/rrds/c3550.rrd
$DATA:$IN:$OUT:$INpk:$OUTpk
Every 5 minute one script create the graphic in
the form of a GIF picture.
To visualize the bit rates we use the following
rrdtool command:
rrdtool graph gigabits20.gif \
--start -86400 -t "Giga 20 sec" \
-w 600 -h 400 --base 1000 \
DEF:i=c3550.rrd:in:AVERAGE \
CDEF:inb=i,8,* \
DEF:o=c3550.rrd:out:AVERAGE \
CDEF:outb=o,8,* \
CDEF:outbn=outb,-1,* \
AREA:inb#00ff88:"IN traf" \
COMMENT:Min \
GPRINT:inb:MIN:%lf%s COMMENT:bps
\
COMMENT:Average \
GPRINT:inb:AVERAGE:%lf%S
COMMENT:bps \
COMMENT:Max \
GPRINT:inb:MAX:%lf%s
COMMENT:bps \
AREA:outbn#ff8800:"Out traff" \
COMMENT:Min \
GPRINT:outb:MIN:%lf%s
COMMENT:bps \
COMMENT:Average \
GPRINT:outb:AVERAGE:%lf%S
COMMENT:bps \
COMMENT:Max \
GPRINT:outb:MAX:%le%s
COMMENT:bps \
HRULE:0#000000
4. Tests
4.1 Connectivity test
Before any exhaustive test could begin we had to
test connectivity between the two switches.
The estimated distance of 110km is slightly over
the 100km range of the product but fortunately after
connecting the optical fiber the connected LED lit
up. CDP also showed the other end switch.
4.2 Layer 3 connectivity test
After some basic configuration a IP connectivity
test could be performed and ping reported 2 ms.
After configuring a routing loop and injecting
some traffic, interface utilization was brought up to
over 95%. Ping reported an average 15 ms and still
with no packet drops. The switch and interfaces are
performing quit well.
4.3 Monitored Heavy load test
50

The Multi-Generator (MGEN) is open source
software which provides the ability to perform IP
network performance tests and measurements using
UDP/IP traffic (TCP is currently being developed).
The toolset generates real-time traffic patterns so
that the network can be loaded in a variety of ways.
Script files are used to drive the generated
loading patterns over time. These script files can be
used to emulate the traffic patterns of unicast and/or
multicast UDP/IP applications.
The main objective of this test is to test the
CWDM fiber link and the c3550 under heavy load.
We have put one static route on each device
pointing each route to the other device for the same
prefix, a typical route loop.
Sending packets with great TTL results in heavy
load and a full 1 Gbit/s traffic between Cluj and Tg-
Mures.
Figure 2 following illustrates the test bed.
Figure 2
We made two heavy load tests using MGEN4.0:
The MGEN generates one UDP stream with TTL
100. This pattern type generates messages of a fixed
(in bytes) at a very regular (in
messages/second). The field must be greater
or equal to the minimum MGEN message size and
less than or equal to the maximum UDP message
size of 8192 bytes.
The first pattern generates messages with fixed
1200.0 bytes and at a regular 1024 rate. After 7200
seconds (2 hours) the stream is modified for a bigger
packet rate, 1500, but the same packet size. Packet
size and rate are shown in the following table for
each time span:
Hour Packet size Packet rate
0 - 2 1200 1024
2 – 4 1200 1500
4 – 6 1200 2000
6 – 8 1200 4096
8 – 10 1600 1024
10 – 12 1600 1500
12 – 14 1600 2000
14 – 16 1600 4096
16 – 18 1900 1024
18 – 20 1900 1500
20 – 22 1900 2000
22 – 24 1900 4096
51

The following results were obtained for a test started at 20:00 hours:
Bytes:
Packets:
52

A second test with an even stronger load was
performed next day. The results were similar and
brought nothing new.
5. Conclusion
The Catalyst 3550 Switch was tested over a dark
fiber link exceeding with 10% the manufacturers
specifications. Wire speed IP switching was
obtained under any circumstances and we couldn’t
detect any loss of performance neither in switching
neither in transmitting the test packets.
6. References
http://www.cisco.com/en/US/customer/products/hw/
switches/ps646/index.html
http://manimac.itd.nrl.navy.mil/MGEN/
http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/
http://www.netperf.org/netperf/NetperfPage.html
http://www.cis.ohio-state.edu/~jain/cis788-
97/gigabit_ethernet/
53