QLogic OFED+ Host Software User Guide, Rev. B

3–TrueScale Cluster Setup and Administration
Performance Settings and Management Tips
On Intel Harpertown CPUs, it may be beneficial to add a
pcie_coalesce=1 parameter to this line.
On AMD CPUs (PCIe Gen1) no ib_qib parameter changes are
recommended.
Alternatively, these PCIe parameters can also be set in the BIOS on some
systems.
• Use PCIe Max Payload size of 256, where available, with the QLE7340,
QLE7342, QLE7240 and QLE7280. The QLE7240 and QLE7280 adapters
can support 128, 256, or 512 bytes. This value is typically set by the BIOS
as the minimum value supported both by the PCIe card and the PCIe root
complex.
• Make sure that write combining is enabled. The x86 Page Attribute Table
(PAT) mechanism that allocates Write Combining (WC) mappings for the
PIO buffers has been added and is now the default. If PAT is unavailable or
PAT initialization fails for some reason, the code will generate a message in
the log and fall back to the MTRR mechanism. See Appendix H Write
Combining for more information.
• Check the PCIe bus width. If slots have a smaller electrical width than
mechanical width, lower than expected performance may occur. Use this
command to check PCIe Bus width:
$ ipath_control -iv
This command also shows the link speed.
• Experiment with non-default CPU affinity while running
single-process-per-node latency or bandwidth benchmarks. Latency
may be slightly lower when using different CPUs (cores) from the default. On
some chipsets, bandwidth may be higher when run from a non-default CPU
or core. See “Performance Tuning” on page 4-23 for more information on
using taskset with QLogic MPI. With another MPI, look at its documentation
to see how to force a benchmark to run with a different CPU affinity than the
default. With OFED micro benchmarks such as from the qperf or perftest
suites, taskset will work for setting CPU affinity.
Turn C-state Off to improve MPI latency (ping-pong) benchmarks on
Nehalem systems. In the BIOS, look for advanced CPU settings, and set the
C-State parameter to "disable."
D000046-005 B 3-27