QLogic OFED+ Host Software User Guide, Rev. B
QLogic OFED+ Host Software User Guide, Rev. B
QLogic OFED+ Host Software User Guide, Rev. B
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
3–TrueScale Cluster Setup and Administration<br />
MPI over uDAPL<br />
• Dbg=7: Debugging<br />
This should only be turned on at the request of <strong>QLogic</strong> Support. This<br />
will generate so much information that system operation will be<br />
impacted. (Includes Dbg=6)<br />
Other Settings<br />
The remaining configuration settings for the Distributed SA are generally only<br />
useful in special circumstances and are not needed in normal operation. The<br />
sample qlogic_sa.conf configuration file contains a brief description of each.<br />
MPI over uDAPL<br />
Intel MPI can be run over uDAPL, which uses InfiniBand Verbs. uDAPL is the user<br />
mode version of the Direct Access Provider Library (DAPL), and is provided as a<br />
part of the OFED packages. You will also have to have IPoIB configured.<br />
The setup for Intel MPI is described in the following steps:<br />
1. Make sure that DAPL 1.2 (not version 2.0) is installed on every node. In this<br />
release they are called compat-dapl. They can be installed either with the<br />
<strong>QLogic</strong> OFED <strong>Host</strong> <strong>Software</strong> package.<br />
2. Verify that there is a /etc/dat.conf file. The file dat.conf contains a list of<br />
interface adapters supported by uDAPL service providers. In particular, it<br />
must contain mapping entries for OpenIB-cma for dapl 1.2.x, in a form<br />
similar to this (all on one line):<br />
OpenIB-cma u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2<br />
"ib0 0" ""<br />
3. On every node, type the following command (as a root user):<br />
# modprobe rdma_ucm<br />
To ensure that the module is loaded when the driver is loaded, add<br />
RDMA_UCM_LOAD=yes to the /etc/infiniband/openib.conf file. (Note that<br />
rdma_cm is also used, but it is loaded automatically.)<br />
4. Bring up an IPoIB interface on every node, for example, ib0. See the<br />
instructions for configuring IPoIB for more details.<br />
For more information on using Intel MPI, see Section 5.<br />
Changing the MTU Size<br />
The Maximum Transfer Unit (MTU) is set to 4K and enabled in the driver by<br />
default. To see the current MTU size, and the maximum supported by the adapter,<br />
type the command:<br />
$ ibv_devinfo<br />
D000046-005 B 3-21
Change language