QLogic OFED+ Host Software User Guide, Rev. B
QLogic OFED+ Host Software User Guide, Rev. B
QLogic OFED+ Host Software User Guide, Rev. B
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
3–TrueScale Cluster Setup and Administration<br />
<strong>QLogic</strong> Distributed Subnet Administration<br />
First, any virtual fabric with a pkey of 0xffff is declared to be the "Default". The<br />
"Default" Virtual Fabric is treated as a special case by the Distributed SA. The<br />
"Default" Virtual Fabric is used only as a last resort. Stored SIDs are only mapped<br />
to the default if they do not match any other Virtual Fabrics. Thus, in the first<br />
example, Figure 3-6, the Distributed SA will assign all the SIDs in its configuration<br />
file to the "PSM_MPI" Virtual Fabric as shown in Figure 3-7.<br />
Virtual Fabric “Default”<br />
Pkey: 0xffff<br />
SID Range: 0x0-0xffffffffffffffff<br />
Virtual Fabric “PSM_MPI”<br />
Pkey: 0x8002<br />
SID Range: 0x1-0xf<br />
SID Range:<br />
0x1000117500000000-<br />
0x100011750000000f<br />
Virtual Fabric “PSM_MPI”<br />
Pkey: 0x8002<br />
SID Range: 0x1-0xf<br />
SID Range: 0x1000117500000000-<br />
0x100011750000000f<br />
Infiniband Fabric<br />
Distributed SA<br />
Figure 3-7. Virtual Fabrics with all SIDs assigned to PSM_MPI Virtual Fabric<br />
Second, the Distributed SA handles overlaps by taking advantage of the fact that<br />
Virtual Fabrics have unique numeric indexes. (These IDs can be seen by using<br />
the command "iba_saquery -o vfinfo".) The Distributed SA will always<br />
assign a SID to the Virtual Fabric with the lowest ID number, as shown in<br />
Figure 3-8. This ensures that all copies of the Distributed SA in the InfiniBand<br />
fabric will make the same decisions about assigning SIDs. However, it also means<br />
that the behavior of your fabric can be affected by the order you configured the<br />
virtual fabrics.<br />
Virtual Fabric “Reserved”<br />
ID: 2 Pkey: 0x8003<br />
SID Range: 0x1-0xf<br />
Virtual Fabric Virtual “Default” Fabric “Default” Pkey: 0xffff<br />
SID Range: Pkey: 0x0-0xffffffffffffffff<br />
0xffff<br />
SID Range:<br />
Virtual<br />
0x0-0xffffffffffffffff<br />
Fabric “PSM_MPI”<br />
ID: 1 Pkey: 0x8002<br />
SID Range: 0x1-0xf<br />
SID Range:<br />
0x1000117500000000-<br />
0x100011750000000f<br />
Infiniband Fabric<br />
Virtual Fabric “PSM_MPI” “Default”<br />
Pkey: 0x8002 0xffff<br />
SID Range: 0x1-0xf<br />
SID Range: 0x1000117500000000-<br />
0x100011750000000f<br />
Distributed SA<br />
Figure 3-8. Virtual Fabrics with Unique Numeric Indexes<br />
3-18 D000046-005 B
Change language