QLogic OFED+ Host Software User Guide, Rev. B

4–Running QLogic MPI on QLogic Adapters
QLogic MPI Details
NOTE:
In rare cases, setting contexts automatically on DDR and QDR InfiniBand
Adapters can lead to sub-optimal performance where one or more
TrueScale hardware contexts have been disabled and a job is run that
requires software context sharing. Since the algorithm ensures that there is
at least one TrueScale context per online CPU, this case occurs only if the
CPUs are over-subscribed with processes (which is not normally
recommended). In this case, it is best to override the default to use as many
TrueScale contexts as are available, which minimizes the amount of
software context sharing required.
Enabling and Disabling Software Context Sharing
By default, context sharing is enabled; it can also be specifically disabled.
Context Sharing Enabled: The MPI library provides PSM the local process
layout so that TrueScale contexts available on each node can be shared if
necessary; for example, when running more node programs than contexts. All
PSM jobs assume that they can make use of all available TrueScale contexts to
satisfy the job requirement and try to give a context to each process.
When context sharing is enabled on a system with multiple QLogic adapter
(TrueScale) boards (units) and the IPATH_UNIT environment variable is set, the
number of TrueScale contexts made available to MPI jobs is restricted to the
number of contexts available on that unit. When multiple TrueScale devices are
present, it restricts the use to a specific TrueScale unit. By default, all configured
units are used in round robin order.
Context Sharing Disabled: Each node program tries to obtain exclusive access
to an TrueScale hardware context. If no hardware contexts are available, the job
aborts.
To explicitly disable context sharing, set this environment variable in one of the
two following ways:
PSM_SHAREDCONTEXTS=0
PSM_SHAREDCONTEXTS=NO
The default value of PSM_SHAREDCONTEXTS is 1 (enabled).
Restricting TrueScale Hardware Contexts
in a Batch Environment
If required for resource sharing between multiple jobs in batch systems, you can
restrict the number of TrueScale hardware contexts that are made available on
each node of an MPI job by setting that number in the
PSM_SHAREDCONTEXTS_MAX environment variable.
D000046-005 B 4-13