Lustre 1.6 Operations Manual

12.3 Using Lustre with Multiple NICs versus
Bonding NICs
Lustre can use multiple NICs without bonding. There is a difference in performance
when Lustre uses multiple NICs versus when it uses bonding NICs.
Whether an aggregated link actually yields a performance improvement
proportional to the number of links provided, depends on network traffic patterns
and the algorithm used by the devices to distribute frames among aggregated links.
Performance with bonding depends on:
■ Out-of-order packet delivery
This can trigger TCP congestion control. To avoid this, some bonding drivers
restrict a single TCP conversation to a single adapter within the bonded group.
■
Load balancing between devices in the bonded group.
Consider a scenario with a two CPU node with two NICs. If the NICs are bonded,
Lustre establishes a single bundle of sockets to each peer. Since ksocklnd bind
sockets to CPUs, only one CPU moves data in and out of the socket for a unidirectional
data flow to each peer. If the NICs are not bonded, Lustre establishes
two bundles of sockets to the peer. Since ksocklnd spreads traffic between sockets,
and sockets between CPUs, both CPUs move data.
12-4 Lustre 1.6 Operations Manual • September 2008