Lustre 1.6 Operations Manual

Credits work like a semaphore. At start they are initialized to allow a certain number
of operations (8 in this example). LNET keeps a track of the minimum value so that
you can see how congested a resource was.
If rtr/tx is less than max, there are operations in progress. The number of
operations is equal to rtr or tx subtracted from max.
If rtr/tx is greater that max, there are operations blocking.
LNET also limits concurrent sends and router buffers allocated to a single peer so
that no peer can occupy all these resources.
/proc/sys/lnet/nis
# cat /proc/sys/lnet/nis
nid refs peer max tx min
0@lo 3 0 0 0 0
192.168.10.34@tcp 4 8 256 256 252
Shows the current queue health on this node. The fields are explained below:
Field
nid
refs
peer
max
tx
min
queue
Description
Network interface
Internal reference counter
Number of peer-to-peer send credits on this NID. Credits are used to
size buffer pools
Total number of send credits on this NID.
Current number of send credits available on this NID.
Lowest number of send credits available on this NID.
Total bytes in active/queued sends.
Subtracting max – tx yields the number of sends currently active. A large or
increasing number of active sends may indicate a problem.
# cat /proc/sys/lnet/nis
nid refs peer max tx min
0@lo 2 0 0 0 0
10.67.73.173@tcp 4 8 256 256 253
20-10 Lustre 1.6 Operations Manual • September 2008