QLogic OFED+ Host Software User Guide, Rev. B
QLogic OFED+ Host Software User Guide, Rev. B
QLogic OFED+ Host Software User Guide, Rev. B
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
4–Running <strong>QLogic</strong> MPI on <strong>QLogic</strong> Adapters<br />
<strong>QLogic</strong> MPI Details<br />
Process Allocation<br />
Normally MPI jobs are run with each node program (process) being associated<br />
with a dedicated <strong>QLogic</strong> infiniband adapter hardware context that is mapped to a<br />
CPU.<br />
If the number of node programs is greater than the available number of hardware<br />
contexts, software context sharing increases the number of node programs that<br />
can be run. Each adapter supports four software contexts per hardware context,<br />
so up to four node programs (from the same MPI job) can share that hardware<br />
context. There is a small additional overhead for each shared context.<br />
Table 4-6 shows the maximum number of contexts available for each adapter.<br />
Table 4-6. Available Hardware and <strong>Software</strong> Contexts<br />
Adapter<br />
Available Hardware<br />
Contexts (same as number<br />
of supported CPUs)<br />
Available Contexts when<br />
<strong>Software</strong> Context Sharing is<br />
Enabled<br />
QLE7140 4 16<br />
QLE7240/<br />
QLE7280<br />
QLE7342/<br />
QLE7340<br />
16 64<br />
16 64<br />
The default hardware context/CPU mappings can be changed on the TrueScale<br />
DDR and QDR InfiniBand Adapters (QLE72x0 and QLE734x). See “TrueScale<br />
Hardware Contexts on the DDR and QDR InfiniBand Adapters” on page 4-12 for<br />
more details.<br />
Context sharing is enabled by default. How the system behaves when context<br />
sharing is enabled or disabled is described in “Enabling and Disabling <strong>Software</strong><br />
Context Sharing” on page 4-13.<br />
When running a job in a batch system environment where multiple jobs may be<br />
running simultaneously, it is useful to restrict the number of TrueScale contexts<br />
that are made available on each node of an MPI. See “Restricting TrueScale<br />
Hardware Contexts in a Batch Environment” on page 4-13.<br />
Errors that may occur with context sharing are covered in “Context Sharing Error<br />
Messages” on page 4-14.<br />
There are multiple ways of specifying how processes are allocated. You can use<br />
the mpihosts file, the -np and -ppn options with mpirun, and the<br />
MPI_NPROCS and PSM_SHAREDCONTEXTS_MAX environment variables. How<br />
these all are set are covered later in this document.<br />
D000046-005 B 4-11
Change language