CUDA Libraries and MPI+OpenMP+CUDA - Prace Training Portal

MPI-GPU binding – Code Snippet (C)/* Create a communicator consisting of the ranks running on this node. */MPI_Comm_group(MPI_COMM_WORLD, &lWorldGroup);MPI_Group_incl(lWorldGroup, lNumRanksThisNode, lRanksThisNode, &lThisNodeGroup);MPI_Comm_create(MPI_COMM_WORLD, lThisNodeGroup, &lThisNodeComm);MPI_Comm_rank(lThisNodeComm, &lRankThisNode);MPI_Comm_size(lThisNodeComm, &lSizeThisNode);/* Attach all MPI processes on this node to the available* GPUs in round-robin fashion*/cudaGetDeviceCount(&lNumDevicesThisNode);/* Multiple MPI can share a GPU... */for (i = 0; i < ngpus_per_process; i++) {qe_gpu_bonded[i] = lRankThisNode % lNumDevicesThisNode;}February 10, 2012 PRACE Winter School 201245