QLogic OFED+ Host Software User Guide, Rev. B
QLogic OFED+ Host Software User Guide, Rev. B
QLogic OFED+ Host Software User Guide, Rev. B
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
4–Running <strong>QLogic</strong> MPI on <strong>QLogic</strong> Adapters<br />
Debugging MPI Programs<br />
To use this feature, the application must be compiled with both OpenMP and MPI<br />
code enabled. To do this, use the -mp flag on the mpicc compile line.<br />
As mentioned previously, MPI routines can be called only by the master OpenMP<br />
thread. The hybrid executable is executed as usual using mpirun, but typically<br />
only one MPI process is run per node and the OpenMP library will create<br />
additional threads to utilize all CPUs on that node. If there are sufficient CPUs on<br />
a node, you may want to run multiple MPI processes and multiple OpenMP<br />
threads per node.<br />
The number of OpenMP threads is typically controlled by the OMP_NUM_THREADS<br />
environment variable in the .mpirunrc file. (OMP_NUM_THREADS is used by<br />
other compilers’ OpenMP products, but is not a <strong>QLogic</strong> MPI environment<br />
variable.) Use this variable to adjust the split between MPI processes and<br />
OpenMP threads. Usually, the number of MPI processes (per node) times the<br />
number of OpenMP threads will be set to match the number of CPUs per node. An<br />
example case would be a node with four CPUs, running one MPI process and four<br />
OpenMP threads. In this case, OMP_NUM_THREADS is set to four.<br />
OMP_NUM_THREADS is on a per-node basis.<br />
See “Environment for Node Programs” on page 4-19 for information on setting<br />
environment variables.<br />
At the time of publication, the MPI_THREAD_SERIALIZED and<br />
MPI_THREAD_MULTIPLE models are not supported.<br />
Debugging MPI Programs<br />
MPI Errors<br />
NOTE:<br />
When there are more threads than CPUs, both MPI and OpenMP<br />
performance can be significantly degraded due to over-subscription of the<br />
CPUs.<br />
Debugging parallel programs is substantially more difficult than debugging serial<br />
programs. Thoroughly debugging the serial parts of your code before parallelizing<br />
is good programming practice.<br />
Almost all MPI routines (except MPI_Wtime and MPI_Wtick) return an error<br />
code; either as the function return value in C functions or as the last argument in a<br />
Fortran subroutine call. Before the value is returned, the current MPI error handler<br />
is called. By default, this error handler aborts the MPI job. Therefore, you can get<br />
information about MPI exceptions in your code by providing your own handler for<br />
MPI_ERRORS_RETURN. See the man page for the MPI_Errhandler_set for<br />
details.<br />
4-26 D000046-005 B
Change language