Dell Power Solutions

HIGH-PERFORMANCE COMPUTINGFigure 1. A typical Rocks cluster layoutas IP address and subnet mask) of the private network NIC. Thesedetails are used to assign IP addresses to the client nodes of thecluster in the latter part of the cluster installation process.Rocks is self-contained, including the necessary tools, scripts,and services for subsequent installation phases. Rocks front-end nodeinstallation takes place on bare-metal hardware. The Rocks CDs installthe OS on the front-end node during the installation procedure.Compute node installationFront-end nodeNIC 1 NIC 2The rest of the Rocks cluster is installed from the front-end node,using a test-based utility called insert-ethers (see Figure 2). To buildthe compute nodes, administrators can launch the insert-ethers utilityfrom the front-end node. The insert-ethers utility is completelydatabase driven. It allows the administrator to configure each nodeas a compute node, to be used for computational purposes; a ParallelVirtual File System (PVFS) 8 node, to be used only for I/O purposes;or a combination compute and PVFS node, to be used for both computationaland I/O purposes.SwitchNode 0 Node 1 Node 2 Node nPublicnetworkThe insert-ethers utility is also used to populate the nodestable in the MySQL database. To this end, the utility continuouslyparses the Linux OS log files (such as the /var/log/messages file)to extract DHCPDISCOVER messages and Media Access Control(MAC) addresses. When the compute node is PXE-booted, the DHCPrequest is sent to the front-end node. When the front-end node findsthe MAC address in the parsed files, and if the MAC address is notpresent in the database, then the front-end node accepts the newnode and installs it using the Rocks kickstart mechanism. Howeverif the MAC address is already present in the database, the frontendnode reinstalls the compute node by assigning the same IPaddress that it previously had. Hence, the insert-ethers utility merelyreimages the compute node by matching previous information presentin the database. Front-end node scripts in Rocks generate thekickstart file on the fly depending on the information—such asarchitecture and CPU count—provided by the compute node throughthe modified Rocks-supplied Red Hat Anaconda installer.Rocks is designed to make compute node installation easy.Rocks can also detect NICs (such as Myricom Myrinet 9 cards)on the compute nodes and compile the correct drivers for theseNICs during installation. Rocks is based on XML and the Python 10programming language. This design, combined with the MySQLdatabase back end, helps provide great flexibility for administratorsto modify compute node settings such as disk partitioning;enable selected services such as Remote Shell (RSH); and installthird-party packages. An administrator can customize the targetnode environment by modifying select XML files in Rocks andsimply re-creating a new Rocks distribution, which can then beused for installation.Rocks cluster management and monitoringRocks uses the basic mechanism of OS reinstallation for updating thenodes in the cluster. If a hardware failure occurs, or if a hard driveor NIC must be replaced, the compute node should be reinstalledfrom the front-end node; the front-end node’s MySQL database isautomatically updated if required. If additional software or updatefiles need to be installed on the compute nodes, then the administratorshould add these to the front-end node and create an updatedRocks distribution. All nodes should then be reinstalled using theupdated distribution to help maintain consistency across the cluster.Reinstallation is fast and helps reduce or eliminate administratortime spent troubleshooting and debugging failed nodes.The Rocks CLI is also used to replace and remove nodes froma cluster. An administrator can create a naming scheme that helpsidentify nodes by their cabinet and rack locations.Step 1: Administrator installs the front-endnode (Linux with Rocks)Step 2: Administrator configures thefront-end nodeStep 3: Administrator launchesthe insert-ethers utilityStep 4: PXE boots the nodeStep 5: Insert-ethers utilitydiscovers the MAC addressStep 6: Rocks starts the OS installationFigure 2. Steps to install a Rocks clusterMySQLdatabaseFront-end nodeCompute node 1Compute node 2PVFS node 3PVFS node 48For more information about PVFS, visit www.parl.clemson.edu/pvfs.9For more information about Myrinet, visit www.myri.com.10For more information about Python, visit www.python.org.www.dell.com/powersolutions Reprinted from Dell Power Solutions, February 2005. Copyright © 2005 Dell Inc. All rights reserved. POWER SOLUTIONS 113