Contents Telektronikk - Telenor

The effect of end system hardware and software on TCP/IP
throughput performance over a local ATM network
BY KJERSTI MOLDEKLEV, ESPEN KLOVNING AND ØIVIND KURE
Abstract
High-speed networks reinstate the
end-system as the communication path
bottleneck. The Internet TCP/IP protocol
suite is the first higher-level protocol
stack to be used on ATM based
networks. In this paper we present
how the host architecture and host network
interface are crucial for memory-to-memory
TCP throughput. In
addition, configurable parameters like
the TCP maximum window size and
the user data size in the write and read
system calls influence the segment flow
and throughput performance. We present
measurements done between
Sparc2 and Sparc10 based machines
for both generations of ATM-adapters
from FORE Systems. The first generation
adapters are based on programmed
I/O; the second generation
adapters on DMA. To explain the variations
in the throughput characteristics,
we put small optimized probes in
the network driver to log the segment
flow on the TCP connections.
1 Introduction
The TCP/IP (Transmission Control Protocol/Internet
Protocol) stack has shown
a great durability. It has been adapted
and widely used over a large variety of
network technologies, ranging from lowspeed
point-to-point lines to high-speed
networks like FDDI (Fiber Distributed
Data Interface) and ATM (Asynchronous
Transfer Mode). The latter is based on
transmission of small fixed-size cells and
aims at offering statistical multiplexing
of connections with different traffic characteristics
and quality of service requirements.
For ATM to work as intended, the
network depends on a characterization of
the data flow on the connection. TCP/IP
has no notion of traffic characteristics
and quality-of-service requirements, and
considers the ATM network as highbandwidth
point-to-point links between
routers and/or end systems. Nevertheless,
TCP/IP is the first protocol to run on top
of ATM.
Several extensions are suggested to make
TCP perform better over networks with a
high bandwidth-delay product [1]. At
present, these extensions are not widely
used. Furthermore, in the measurements
to be presented in this paper, the propagation
delay is minimal making the
extensions above of little importance.
The TCP/IP protocol stack, and in particular
its implementations for BSD UNIX
derived operating systems, has continuously
been a topic for analyses [2], [3],
[4], [5], [6], [7]. These analyses consider
networks with lower bandwidth or smaller
frame transmission units than the cellbased
ATM network can offer through
the ATM adaptation layers, AALs.
This paper contributes to the TCP analyses
along two axes. The first is the
actual throughput results of TCP/IP over
a high-speed local area ATM network for
popular host network interfaces and host
architectures. Measurements are done on
both Sparc2 and Sparc10 based machines
using both generations of ATM network
interfaces from FORE Systems; the programmed
I/O based SBA-100 adapters
with segmentation and reassembly in network
driver software, and the more
advanced DMA based SBA-200 adapters
with on-board segmentation and reassembly.
Both the hardware and software
components of the network interface,
the network adapter and the network
driver, respectively, are upgraded
between our different measurements.
The second axis is an analysis of how
and why the hardware and software components
influence the TCP/IP segment
flow and thereby the measured performance.
The software parameters with the
largest influence are the maximum window
size and the user data size. In general,
the throughput increases with increasing
window and user data sizes up
to certain limits. It is not a monotonous
behavior; the throughput graphs have
their peaks and drops.
TCP is byte-stream oriented [8]. The segmentation
of the byte stream depends on
the user data size, the window flow control,
the acknowledgment scheme, an
algorithm (Nagle’s) to avoid the transmission
of many small segments, and the
operating system integration of the TCP
implementation. The functionality and
speed of the host and network interface
also influence the performance; more
powerful machines and more advanced
interfaces can affect the timing relationships
between data segments and window
updates and acknowledgments.
The rest of this paper is outlined as follows:
The next section describes the
measurement environment and methods.
The third section presents in more detail
the software and hardware factors influencing
the performance; the protocol
mechanisms, the system environment
factors, and the host architecture. The
fourth section contains throughput
measurements and segment flow analysis
of our reference architecture, a Sparc2
based machine using the programmed
I/O based SBA-100 interface. The fifth
section presents performance results
when upgrading a software component,
namely the network driver of the network
interface. The sixth section discusses the
results when upgrading the hosts to
Sparc10 based machines. The throughput
results and segment flows using the
SBA-200 adapters in Sparc10 machines
follow in the seventh section. The paper
closes with summary and conclusions.
2 Measurement environment
and methods
The performance measurements in this
paper are based on the standard TCP/IP
protocol stack in SunOS 4.1.x. We used
two Sparc2 based Sun IPX machines and
two Sparc10 based Axil 311/5.1
machines. The I/O bus of both machine
architectures is the Sbus [9] to which the
network adapter is attached. The Sun
machines run SunOS 4.1.1, while the
Axil machines run SunOS 4.1.3. For our
TCP measurements the differences
between the two SunOS versions are
negligible. Access to the TCP protocol is
through the BSD-based socket interface
[11]. The workstations have both ATM
and ethernet network connections. Figure
1 illustrates the measurement environment
and set-up.
2.1 The local ATM network
The workstations are connected to an
ATM switch, ASX-100, from FORE Systems.
The ASX-100 is a 2.5 Gbit/s busbased
ATM switch with an internal
Sparc2 based switch controller. The
ATM physical interface is a 140 Mbit/s
TAXI interface [12]. The ATM host network
interfaces, SBA-100 and SBA-200,
are the first and second generation from
FORE.
The first generation Sbus ATM adapter,
SBA-100 [17], [18], is a simple slaveonly
interface based on programmed I/O.
The ATM interface has a 16 kbyte
receive FIFO and a 2 kbyte transmit
FIFO. The SBA-100 network adapter
performs on-board computation of the
cell based AAL3/4 CRC, but the segmentation
and reassembly between
frames and cells are done entirely in software
by the network driver. The SBA-
100 adapters have no hardware support
for AAL5 frame based CRC. Therefore,
using the AAL3/4 adaptation layer gives
the best performance. The SBA-100
adapters were configured to issue an
155