Lustre 1.6 Operations Manual

To obtain the optimal performance for your platform, you may want to vary the
remaining options.
n_waitd (1) sets the number of threads that process completed MX requests (sends
and receives).
max_peers (1024) tells MXLND the upper limit of machines that it will need to
communicate with. This affects how many receives it will pre-post and each receive
will use one page of memory. Ideally, on clients, this value will be equal to the total
number of Lustre servers (MDS and OSS). On servers, it needs to equal the total
number of machines in the storage system. cksum (0) turns on small message
checksums. It can be used to aid in troubleshooting. MX also provides an optional
checksumming feature which can check all messages (large and small). For details,
see the MX README.
ntx (256) is the number of total sends in flight from this machine. In actuality,
MXLND reserves half of them for connect messages so make this value twice as
large as you want for the total number of sends in flight.
credits (8) is the number of in-flight messages for a specific peer. This is part of the
flow-control system in Lustre. Increasing this value may improve performance but it
requires more memory because each message requires at least one page.
board (0) is the index of the Myricom NIC. Hosts can have multiple Myricom NICs
and this identifies which one MXLND should use. This value must match the board
value in your MXLND hosts file for this host.
ep_id (3) is the MX endpoint ID. Each process that uses MX is required to have at
least one MX endpoint to access the MX library and NIC. The ID is a simple index
starting at zero (0). This value must match the endpoint ID value in your MXLND
hosts file for this host.
polling (0) determines whether this host will poll or block for MX request
completions. A value of 0 blocks and any positive value will poll that many times
before blocking. Since polling increases CPU usage, we suggest that you set this to
zero (0) on the client and experiment with different values for servers.
Chapter 31 Configuration Files and Module Parameters (man5) 31-21