Lustre 1.6 Operations Manual

Information on the Lustre Networking (LNET) protocol
Lustre layers the socket LND (socklnd) protocol above TCP/IP. Every LNET
message is an lnet_hdr_t sent in (little-endian (LE) byte order followed by
'payload_length' bytes of opaque payload data. There are four types of messages.
■
■
■
■
PUT - request to send data contained in the payload
ACK - response to a PUT with ack_wmd != LNET_WIRE_HANDLE_NONE
GET - request to fetch data
REPLY - response to a GET with data in the payload
Typically, ACK and GET messages have 0 bytes of payload.
Explanation of: '... previously skipped # similar messages' in Lustre logs
Unlike syslog, which occupies exactly identical lines, the space for Lustre messages
is occupied if there are bursts of messages from the same line of code, even if they
are not sequential. This avoids duplication of the same event from different clients,
or in cases where two or more messages are repeated.
All messages are kept in the Lustre kernel debug log, so "lctl dk" at that time would
show all messages (in case they are not wrapped).
Printing a large number of messages to the kernel console can dramatically slow
down the system. As this happens with IRQs disabled and for a slow console, it
severely impacts overall system performance when there are large number of
messages.
For example:
LustreError: 559:0:(genops.c:1292:obd_export_evict_by_nid())
evicting
b155f37b-b426-ccc2-f0a9-bfbf00000000 at adminstrative request
LustreError: 559:0:(genops.c:1292:obd_export_evict_by_nid())
previously skipped 2 similar messages
In this case, the 'similar' messages are reported for the exact line of source, without
matching the text. Therefore, this is expected output for evictions of more than one
client.
Appendix D Lustre Knowledge Base D-29