The IBM eServer BladeCenter JS20 - IBM Redbooks
The IBM eServer BladeCenter JS20 - IBM Redbooks
The IBM eServer BladeCenter JS20 - IBM Redbooks
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>The</strong> <strong>IBM</strong> Eserver<br />
<strong>BladeCenter</strong> <strong>JS20</strong><br />
Learn about <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
and blade server technology<br />
Understand the PowerPC 970<br />
microprocessor architecture<br />
Plan for, install, and<br />
configure <strong>BladeCenter</strong> <strong>JS20</strong><br />
ibm.com/redbooks<br />
Ben Gibbs<br />
Omkhar Arasaratnam<br />
Robert Arenburg<br />
Bradley Elkin<br />
Rogeli Grima<br />
James Kelly
International Technical Support Organization<br />
<strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
June 2005<br />
SG24-6342-01
Note: Before using this information and the product it supports, read the information in<br />
“Notices” on page xi.<br />
Second Edition (June 2005)<br />
This edition applies to the <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>, type 8842.<br />
© Copyright International Business Machines Corporation 2005. All rights reserved.<br />
Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP<br />
Schedule Contract with <strong>IBM</strong> Corp.
Contents<br />
Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii<br />
Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix<br />
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi<br />
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii<br />
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii<br />
<strong>The</strong> team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii<br />
Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv<br />
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv<br />
Part 1. Introduction to the <strong>BladeCenter</strong> <strong>JS20</strong>. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1<br />
Chapter 1. Introduction to <strong>BladeCenter</strong> and blade server technology. . . . 3<br />
Chapter 2. Hardware components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7<br />
2.1 Overview of the <strong>BladeCenter</strong> infrastructure . . . . . . . . . . . . . . . . . . . . . . . . 8<br />
2.1.1 <strong>BladeCenter</strong> chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8<br />
2.1.2 <strong>BladeCenter</strong> power options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10<br />
2.1.3 Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13<br />
2.1.4 I/O modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16<br />
2.1.5 Blade servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21<br />
2.2 <strong>BladeCenter</strong> <strong>JS20</strong> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22<br />
2.2.1 Base features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24<br />
2.2.2 Optional features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27<br />
2.3 PowerPC 970 and PowerPC 970FX Microprocessors . . . . . . . . . . . . . . . 29<br />
2.3.1 Review of POWER and PowerPC Architecture . . . . . . . . . . . . . . . . 30<br />
2.3.2 Vector/SIMD Multimedia eXtension . . . . . . . . . . . . . . . . . . . . . . . . . 38<br />
2.3.3 Features of the PowerPC 970 and PowerPC 970FX . . . . . . . . . . . . 42<br />
Chapter 3. Software environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47<br />
3.1 Operating system support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48<br />
3.1.1 AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48<br />
3.1.2 Red Hat Enterprise Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48<br />
3.1.3 SUSE LINUX Enterprise Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49<br />
3.2 System management tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49<br />
3.2.1 <strong>BladeCenter</strong> Web interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49<br />
3.2.2 <strong>IBM</strong> Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50<br />
3.2.3 <strong>IBM</strong> Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . 50<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. iii
Chapter 4. Planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53<br />
4.1 Network planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54<br />
4.1.1 Minimal network requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54<br />
4.1.2 High-performance, low-latency network requirements . . . . . . . . . . . 58<br />
4.1.3 Multiple <strong>BladeCenter</strong> environments . . . . . . . . . . . . . . . . . . . . . . . . . 59<br />
4.2 Operating system installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60<br />
4.2.1 Network installation planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60<br />
4.3 Systems management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63<br />
4.3.1 <strong>IBM</strong> Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63<br />
4.3.2 <strong>IBM</strong> Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . 64<br />
Part 2. Implementing the <strong>BladeCenter</strong> <strong>JS20</strong>. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67<br />
Chapter 5. Hardware setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69<br />
5.1 Management Module configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70<br />
5.2 LAN Switch I/O Module configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73<br />
5.3 Blade server configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76<br />
5.3.1 Assigning names to blade servers . . . . . . . . . . . . . . . . . . . . . . . . . . 77<br />
5.3.2 Setting the boot sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78<br />
5.4 Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79<br />
5.4.1 Management Module firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80<br />
5.4.2 LAN Switch I/O Module firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . 82<br />
5.4.3 <strong>BladeCenter</strong> <strong>JS20</strong> firmware (BIOS) . . . . . . . . . . . . . . . . . . . . . . . . . 84<br />
5.4.4 Integrated Systems Management Processor firmware . . . . . . . . . . . 85<br />
5.5 Providing a console for the <strong>JS20</strong> blades . . . . . . . . . . . . . . . . . . . . . . . . . . 87<br />
5.5.1 Configuring Serial over LAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89<br />
5.5.2 Using Serial over LAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91<br />
5.5.3 Open Firmware interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94<br />
5.5.4 Specifying IP parameters to Open Firmware . . . . . . . . . . . . . . . . . . 96<br />
Chapter 6. Installing Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101<br />
6.1 Installing Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102<br />
6.1.1 Configuring the sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102<br />
6.1.2 <strong>The</strong> zImage.initrd file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107<br />
6.2 Configuring BOOTP and TFTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109<br />
6.2.1 BOOTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110<br />
6.2.2 Trivial File Transfer Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111<br />
6.3 Preparing an unattended install. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111<br />
6.3.1 Red Hat Kickstart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112<br />
6.3.2 SuSE AutoYaST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114<br />
6.4 Performing an unattended installation. . . . . . . . . . . . . . . . . . . . . . . . . . . 116<br />
6.4.1 Open Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117<br />
6.4.2 mkzimage_cmdline: SuSE only. . . . . . . . . . . . . . . . . . . . . . . . . . . . 117<br />
6.5 Performing a network installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118<br />
iv <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
6.5.1 SUSE LINUX Enterprise Server 9. . . . . . . . . . . . . . . . . . . . . . . . . . 118<br />
6.5.2 Red Hat Enterprise Linux 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121<br />
Chapter 7. Installing AIX on the <strong>JS20</strong> . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123<br />
7.1 Minimal NIM installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124<br />
7.2 Adding resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127<br />
7.3 Adding machine information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130<br />
7.4 Preparing the NIM master . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134<br />
7.5 Preparing the client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135<br />
Chapter 8. System management scenarios . . . . . . . . . . . . . . . . . . . . . . . 139<br />
8.1 <strong>IBM</strong> Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140<br />
8.1.1 Setting up an <strong>IBM</strong> Director Server . . . . . . . . . . . . . . . . . . . . . . . . . 142<br />
8.1.2 Installing an <strong>IBM</strong> Director Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . 145<br />
8.1.3 Using Director to manage <strong>JS20</strong>s . . . . . . . . . . . . . . . . . . . . . . . . . . 147<br />
8.2 <strong>IBM</strong> Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147<br />
8.2.1 Setting up a CSM management node . . . . . . . . . . . . . . . . . . . . . . . 148<br />
8.2.2 Installing and managing <strong>BladeCenter</strong> <strong>JS20</strong>s using CSM . . . . . . . . 154<br />
Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159<br />
Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161<br />
<strong>IBM</strong> <strong>Redbooks</strong> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161<br />
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161<br />
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162<br />
How to get <strong>IBM</strong> <strong>Redbooks</strong> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164<br />
Help from <strong>IBM</strong> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164<br />
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165<br />
Contents v
vi <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Figures<br />
2-1 <strong>BladeCenter</strong> chassis front view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8<br />
2-2 <strong>BladeCenter</strong> chassis rear view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9<br />
2-3 <strong>BladeCenter</strong> power domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11<br />
2-4 KVM Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14<br />
2-5 Management Module connectors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15<br />
2-6 <strong>BladeCenter</strong> <strong>JS20</strong> physical layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22<br />
2-7 <strong>BladeCenter</strong> <strong>JS20</strong> block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23<br />
2-8 PowerPC logical processing model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34<br />
2-9 PowerPC registers used by application programs . . . . . . . . . . . . . . . . . 35<br />
2-10 Scalar and vector operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39<br />
2-11 PowerPC with VMX logical processing model . . . . . . . . . . . . . . . . . . . . 40<br />
2-12 VMX registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41<br />
2-13 PowerPC 970 internal organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43<br />
4-1 Network logical view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55<br />
4-2 Myrinet network infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59<br />
5-1 Management Module controls and indicators . . . . . . . . . . . . . . . . . . . . 71<br />
5-2 IP address configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72<br />
5-3 LAN Switch I/O Module IP address . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74<br />
5-4 LAN Switch I/O Module setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75<br />
5-5 Blade server naming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77<br />
5-6 Boot sequence of blades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78<br />
5-7 Viewing firmware levels on <strong>BladeCenter</strong> . . . . . . . . . . . . . . . . . . . . . . . . 80<br />
5-8 Updating the Management Module firmware . . . . . . . . . . . . . . . . . . . . . 81<br />
5-9 Firmware upgrade of 4-Port Gb Ethernet Switch Module . . . . . . . . . . . 83<br />
5-10 Blade firmware update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86<br />
5-11 Serial over LAN components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88<br />
5-12 Serial over LAN configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90<br />
5-13 Serial over LAN status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91<br />
5-14 Management Module’s command line interface . . . . . . . . . . . . . . . . . . 92<br />
5-15 Setting CRLF in Windows 2000 Telnet . . . . . . . . . . . . . . . . . . . . . . . . . 94<br />
5-16 POST checkpoint codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95<br />
6-1 YaST installation server configuration . . . . . . . . . . . . . . . . . . . . . . . . . 104<br />
6-2 Selecting the distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105<br />
6-3 Source Configuration pane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106<br />
6-4 Completing the setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107<br />
6-5 redhat-config-kickstart main window . . . . . . . . . . . . . . . . . . . . . . . . . . 112<br />
6-6 AutoYaST module within YaST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114<br />
6-7 Main AutoYaST window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. vii
6-8 Language selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119<br />
6-9 Main menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119<br />
6-10 Installation menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120<br />
6-11 Source menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120<br />
6-12 Protocol menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120<br />
6-13 Installation location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121<br />
7-1 NIM configuration window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124<br />
7-2 Configuring as a NIM Master . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125<br />
7-3 Selecting the NIM interface and network name . . . . . . . . . . . . . . . . . . 125<br />
7-4 Choosing to initialize minimal NIM environment . . . . . . . . . . . . . . . . . 126<br />
7-5 Finalizing the NIM environment settings . . . . . . . . . . . . . . . . . . . . . . . 126<br />
7-6 lpp_source and SPOT configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 127<br />
7-7 AIX source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128<br />
7-8 Default resource settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128<br />
7-9 Completing the resource installation . . . . . . . . . . . . . . . . . . . . . . . . . . 129<br />
7-10 Adding new machine information to NIM . . . . . . . . . . . . . . . . . . . . . . . 130<br />
7-11 Host name of new machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131<br />
7-12 Entering machine-specific information . . . . . . . . . . . . . . . . . . . . . . . . . 132<br />
7-13 Entering MAC address information . . . . . . . . . . . . . . . . . . . . . . . . . . . 133<br />
7-14 Installing an operating system from NIM . . . . . . . . . . . . . . . . . . . . . . . 134<br />
7-15 Setting BOS preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135<br />
7-16 Installation language panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136<br />
7-17 Operating system selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136<br />
7-18 Installation and language options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137<br />
7-19 Kernel options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138<br />
8-1 Selecting the type of database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143<br />
8-2 Database connection information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144<br />
8-3 Setting Director server encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145<br />
8-4 Installing the Director Agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146<br />
8-5 Configuring Director Agent security . . . . . . . . . . . . . . . . . . . . . . . . . . . 146<br />
viii <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Tables<br />
2-1 Power domain A minimum power requirements . . . . . . . . . . . . . . . . . . 12<br />
2-2 Power domain B minimum power requirements . . . . . . . . . . . . . . . . . . 13<br />
2-3 Physical attributes of PowerPC 970 and PowerPC 970FX . . . . . . . . . . 42<br />
5-1 <strong>BladeCenter</strong> and <strong>BladeCenter</strong> <strong>JS20</strong> firmware components . . . . . . . . . 79<br />
5-2 Management Module commands for SoL . . . . . . . . . . . . . . . . . . . . . . . 93<br />
5-3 Network configuration information worksheet . . . . . . . . . . . . . . . . . . . . 96<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. ix
x <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Notices<br />
This information was developed for products and services offered in the U.S.A.<br />
<strong>IBM</strong> may not offer the products, services, or features discussed in this document in other countries. Consult<br />
your local <strong>IBM</strong> representative for information on the products and services currently available in your area.<br />
Any reference to an <strong>IBM</strong> product, program, or service is not intended to state or imply that only that <strong>IBM</strong><br />
product, program, or service may be used. Any functionally equivalent product, program, or service that<br />
does not infringe any <strong>IBM</strong> intellectual property right may be used instead. However, it is the user's<br />
responsibility to evaluate and verify the operation of any non-<strong>IBM</strong> product, program, or service.<br />
<strong>IBM</strong> may have patents or pending patent applications covering subject matter described in this document.<br />
<strong>The</strong> furnishing of this document does not give you any license to these patents. You can send license<br />
inquiries, in writing, to:<br />
<strong>IBM</strong> Director of Licensing, <strong>IBM</strong> Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.<br />
<strong>The</strong> following paragraph does not apply to the United Kingdom or any other country where such provisions<br />
are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES<br />
THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,<br />
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,<br />
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer<br />
of express or implied warranties in certain transactions, therefore, this statement may not apply to you.<br />
This information could include technical inaccuracies or typographical errors. Changes are periodically made<br />
to the information herein; these changes will be incorporated in new editions of the publication. <strong>IBM</strong> may<br />
make improvements and/or changes in the product(s) and/or the program(s) described in this publication at<br />
any time without notice.<br />
Any references in this information to non-<strong>IBM</strong> Web sites are provided for convenience only and do not in any<br />
manner serve as an endorsement of those Web sites. <strong>The</strong> materials at those Web sites are not part of the<br />
materials for this <strong>IBM</strong> product and use of those Web sites is at your own risk.<br />
<strong>IBM</strong> may use or distribute any of the information you supply in any way it believes appropriate without<br />
incurring any obligation to you.<br />
Information concerning non-<strong>IBM</strong> products was obtained from the suppliers of those products, their published<br />
announcements or other publicly available sources. <strong>IBM</strong> has not tested those products and cannot confirm<br />
the accuracy of performance, compatibility or any other claims related to non-<strong>IBM</strong> products. Questions on<br />
the capabilities of non-<strong>IBM</strong> products should be addressed to the suppliers of those products.<br />
This information contains examples of data and reports used in daily business operations. To illustrate them<br />
as completely as possible, the examples include the names of individuals, companies, brands, and products.<br />
All of these names are fictitious and any similarity to the names and addresses used by an actual business<br />
enterprise is entirely coincidental.<br />
COPYRIGHT LICENSE:<br />
This information contains sample application programs in source language, which illustrates programming<br />
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in<br />
any form without payment to <strong>IBM</strong>, for the purposes of developing, using, marketing or distributing application<br />
programs conforming to the application programming interface for the operating platform for which the<br />
sample programs are written. <strong>The</strong>se examples have not been thoroughly tested under all conditions. <strong>IBM</strong>,<br />
therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,<br />
modify, and distribute these sample programs in any form without payment to <strong>IBM</strong> for the purposes of<br />
developing, using, marketing, or distributing application programs conforming to <strong>IBM</strong>'s application<br />
programming interfaces.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. xi
Trademarks<br />
<strong>The</strong> following terms are trademarks of the International Business Machines Corporation in the United States,<br />
other countries, or both:<br />
Eserver®<br />
Eserver®<br />
developerWorks®<br />
<strong>eServer</strong><br />
ibm.com®<br />
iSeries<br />
pSeries®<br />
xSeries®<br />
AIX 5L<br />
AIX®<br />
AS/400®<br />
xii <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
<strong>BladeCenter</strong><br />
Domino®<br />
<strong>IBM</strong>®<br />
Lotus®<br />
PowerPC Architecture<br />
PowerPC 601®<br />
PowerPC 603<br />
PowerPC 604<br />
PowerPC®<br />
POWER<br />
POWER2<br />
<strong>The</strong> following terms are trademarks of other companies:<br />
POWER3<br />
POWER4<br />
POWER4+<br />
POWER5<br />
<strong>Redbooks</strong><br />
<strong>Redbooks</strong> (logo) <br />
RETAIN®<br />
RS/6000®<br />
ServeRAID<br />
System/360<br />
3090<br />
Intel, Intel Inside (logos), MMX, and Pentium are trademarks of Intel Corporation in the United States, other<br />
countries, or both.<br />
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the<br />
United States, other countries, or both.<br />
UNIX is a registered trademark of <strong>The</strong> Open Group in the United States and other countries.<br />
Other company, product, and service names may be trademarks or service marks of others.
Preface<br />
Blade servers are a relatively new technology. <strong>The</strong>y have captured industry focus<br />
because of their modular design, which can reduce cost with a more efficient use<br />
of valuable floor space. <strong>The</strong>y offer simplified management, which can help to<br />
speed such tasks as installing, reprovisioning, updating, and troubleshooting<br />
hundreds of blade servers. You can do all of this remotely using one graphical<br />
console with <strong>IBM</strong>® Director systems management tools.<br />
In addition, blade servers provide improved performance by doubling current rack<br />
density. By integrating resources and sharing key components, costs decrease<br />
and availability increases.<br />
<strong>The</strong> <strong>IBM</strong> Eserver® <strong>BladeCenter</strong> boasts innovative modular technology,<br />
leadership density, and availability. It was designed to help solve a multitude of<br />
real-world problems.<br />
This <strong>IBM</strong> Redbook takes an in-depth look at the <strong>IBM</strong> Eserver <strong>BladeCenter</strong><br />
<strong>JS20</strong>. This is a two-way blade server for applications requiring 64-bit computing.<br />
It is ideal for computer-intensive applications and transactional Internet servers.<br />
This <strong>IBM</strong> Redbook helps you to install, tailor, and configure the <strong>IBM</strong> Eserver<br />
<strong>BladeCenter</strong> <strong>JS20</strong>.<br />
<strong>The</strong> team that wrote this redbook<br />
This redbook was produced by a team of specialists from around the world<br />
working at the International Technical Support Organization (ITSO), Austin<br />
Center.<br />
Ben Gibbs is a Senior Consulting Engineer with Technonics, Inc.<br />
(http://www.technonics.com) in Austin, Texas. He has over 20 years of<br />
experience with UNIX®-based operating systems. He started working with the<br />
AIX® operating system in November of 1989. His areas of expertise include<br />
performance analysis and tuning, operating system internals, and device driver<br />
development for the AIX operating system. He is also an <strong>IBM</strong> Learning Services<br />
instructor for advanced AIX classes. He was the project leader for this <strong>IBM</strong><br />
Redbook.<br />
Omkhar Arasaraknum is a Team Leader with <strong>IBM</strong> Global Services in Canada.<br />
He has worked with Linux since 1998 and uses SuSE, Red Hat, and Gentoo.<br />
Omkhar is the Linux technical lead within his Service Delivery Center and has<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. xiii
worked actively with Linux adoption within <strong>IBM</strong>. Omkhar has also written <strong>IBM</strong><br />
Redpapers about Microsoft® Windows® to Linux migrations for <strong>IBM</strong>.<br />
Robert Arenburg is a Senior Technical Consultant in the Solutions Enablement<br />
organization in the <strong>IBM</strong> Systems and Technology Group and is located in Austin,<br />
Texas. He has a PhD in Engineering Mechanics from Virginia Tech. He has<br />
worked at <strong>IBM</strong> for 13 years. His areas of expertise include high performance<br />
computing, performance, capacity planning, 3D graphics, solid mechanics,<br />
computational, and finite element methods.<br />
Bradley Elkin is a Senior Software Engineer for <strong>IBM</strong> U.S. He holds a PhD in<br />
Chemical Engineering from the University of Pennsylvania and has 17 years of<br />
experience in high performance computing. His areas of expertise include<br />
applications in computational fluid mechanics, computational chemistry, and<br />
bioinformatics. He has written several articles for <strong>IBM</strong> ^ Development<br />
Domain.<br />
Rogeli Grima is a research staff member at the CEPBA <strong>IBM</strong> Research Institute<br />
in Spain. He has three years of experience in applied mathematics. He has also<br />
collaborated on the development of <strong>BladeCenter</strong> <strong>JS20</strong> solution for<br />
bioinformatics.<br />
James Kelly is a Systems Architect in <strong>IBM</strong> Australia specializing in deep<br />
computing. He has 17 years of experience in system engineering and technical<br />
sales support. His current areas of expertise include the solution design,<br />
benchmarking, and installation of server clusters running either AIX or Linux, to<br />
support the installation of high performance computing applications.<br />
Thanks to the following people for their contributions to this project:<br />
ITSO, Austin Center<br />
Trina Bunting<br />
<strong>IBM</strong> Dallas<br />
Luciano Chavez<br />
Matt Davis<br />
Martin Gramlich<br />
Jon Mason<br />
Doug W Oliver<br />
Brian Rzycki<br />
<strong>IBM</strong> Austin<br />
Anton Blanchard<br />
<strong>IBM</strong> OzLabs, Canberra, Australia<br />
xiv <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Janet Ellsworth<br />
<strong>IBM</strong> Poughkeepsie, New York<br />
Become a published author<br />
Join us for a two- to six-week residency program! Help write an <strong>IBM</strong> Redbook<br />
dealing with specific products or solutions, while getting hands-on experience<br />
with leading-edge technologies. You'll team with <strong>IBM</strong> technical professionals,<br />
Business Partners and/or customers.<br />
Your efforts will help increase product acceptance and customer satisfaction. As<br />
a bonus, you'll develop a network of contacts in <strong>IBM</strong> development labs, and<br />
increase your productivity and marketability.<br />
Find out more about the residency program, browse the residency index, and<br />
apply online at:<br />
Comments welcome<br />
ibm.com/redbooks/residencies.html<br />
Your comments are important to us!<br />
We want our <strong>Redbooks</strong> to be as helpful as possible. Send us your comments<br />
about this or other <strong>Redbooks</strong> in one of the following ways:<br />
► Use the online Contact us review redbook form found at:<br />
ibm.com/redbooks<br />
► Send your comments in an Internet note to:<br />
redbook@us.ibm.com<br />
► Mail your comments to:<br />
<strong>IBM</strong> Corporation, International Technical Support Organization<br />
Dept. JN9B Building 905 Internal Zip 9053D003<br />
11501 Burnet Road<br />
Austin, Texas 78758-3493<br />
Preface xv
xvi <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Part 1 Introduction<br />
Part 1<br />
to the<br />
<strong>BladeCenter</strong> <strong>JS20</strong><br />
This part introduces and describes the hardware and software components of<br />
the <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>. It also presents planning considerations.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 1
2 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
1<br />
Chapter 1. Introduction to <strong>BladeCenter</strong><br />
and blade server technology<br />
This chapter presents an overview of <strong>IBM</strong> Eserver <strong>BladeCenter</strong> and blade<br />
server technology. It also covers the hardware for the chassis and the blades.<br />
<strong>The</strong> term blade server refers to a chassis that can hold a number of<br />
hot-swappable devices called blades. Blades come in two varieties: server blades<br />
and option blades.<br />
A server blade is an independent server that contains one or more processors<br />
and associated memory, disk storage, and network controllers. It runs its own<br />
operating system and applications. Each server blade within a system chassis<br />
slides into a blade bay2. It plugs into a midplane or backplane to share common<br />
infrastructure components. <strong>The</strong>se components may include power supplies, fans,<br />
CD-ROM, and floppy drives, Ethernet and Fibre Channel switches, and system<br />
ports.<br />
Option blades may be sharable by server blades. Option blades provide<br />
additional features, such as controllers for external I/O or disk arrays, power<br />
supplies, and so on.<br />
For organizations that seek server consolidation, <strong>BladeCenter</strong> centralizes<br />
servers for increased flexibility, ease of maintenance, reduced cost, and<br />
streamlined human resources. Companies that need to install new applications<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 3
for e-commerce and On Demand Business can achieve speed while ensuring<br />
flexibility, scalability, and availability. For such enterprise requirements as<br />
file-and-print and collaboration, <strong>BladeCenter</strong> is designed to offer reliability,<br />
flexibility for growth, and cost effectiveness. And clients with compute-intensive<br />
applications that need highly available clustering can use the <strong>BladeCenter</strong> to<br />
help achieve high degrees of scalability and performance.<br />
Benefits of the <strong>IBM</strong> Eserver <strong>BladeCenter</strong> family<br />
<strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> family of products features a modular design that<br />
integrates multiple computing resources into a cost-effective, high-density<br />
enclosure for a platform that provides:<br />
► Lower cost in certain configurations: Due to efficiencies in power usage, heat<br />
emissions, and data center floor space utilization, when a configuration of<br />
other servers exceeds one rack, it is often less expensive over a multi-year<br />
period to use a single rack of <strong>BladeCenter</strong> chassis and <strong>BladeCenter</strong> <strong>JS20</strong>s.<br />
► Physical server consolidation: A <strong>BladeCenter</strong> server is an ideal way to<br />
replace many uniprocessor or 2-way servers to save space. 14U of rack<br />
space for 14 1U servers can be replaced with one 7U <strong>BladeCenter</strong> chassis.<br />
Additional rack space that is normally taken for Ethernet, Fibre Channel, and<br />
other switches is eliminated by the integrated switches in the <strong>BladeCenter</strong><br />
chassis.<br />
► High availability: <strong>The</strong> <strong>BladeCenter</strong> chassis offers redundant and hot-swap<br />
components that can prevent failure of the chassis or blade servers when one<br />
of those components fail. <strong>The</strong> chassis has redundant, hot-swap cooling,<br />
power, midplane features, management, and I/O switches. It also has a<br />
hot-swap media tray, with CD-ROM and floppy drive, that can be removed<br />
and serviced while all blade servers are operating.<br />
► Integrated switch technology: All service is from the front and rear of<br />
<strong>BladeCenter</strong>. <strong>The</strong>re is no need to slide the chassis out of the rack and remove<br />
the top cover for service. Also, numerous cables are eliminated, reducing<br />
both cabling cost and servicer or administrator time.<br />
► More integrated systems management features: <strong>The</strong> <strong>BladeCenter</strong> chassis<br />
includes a management module. This module eliminates the need for<br />
individual management adapters, such as the Remote Supervisor Adapter<br />
I or II, in each blade server for remote control. It also eliminates the need for<br />
RS485 interconnect cabling between the blade servers.<br />
► SAN optimization: A Fibre Channel switch module installed in a <strong>BladeCenter</strong><br />
chassis provides storage area network (SAN) access to all blade servers in<br />
the chassis without internal cabling.<br />
4 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
<strong>BladeCenter</strong> versus an <strong>IBM</strong> rack server<br />
<strong>BladeCenter</strong> is an ideal solution for certain environments. In other environments,<br />
an <strong>IBM</strong> rack server may be a better fit:<br />
► Need for a small number of servers: A <strong>BladeCenter</strong> chassis is required for one<br />
blade to be cost effective compared to some stand-alone servers. <strong>The</strong>refore,<br />
a chassis should be full or nearly full.<br />
► Need for existing Peripheral Component Interconnect (PCI) adapters:<br />
<strong>BladeCenter</strong> does not include adapter slots as shipped. An optional I/O<br />
Expansion Card feature offers one PCI slot per blade, limiting the blade to one<br />
integrated development environment (IDE) drive. <strong>The</strong> expansion feature<br />
supports PCI cards designed for <strong>BladeCenter</strong>. <strong>The</strong>refore, a mix of<br />
<strong>BladeCenter</strong> and traditional rack-optimized servers may be appropriate.<br />
► Need for large internal storage or external SCSI storage: <strong>BladeCenter</strong><br />
supports a maximum of 80 GB of IDE or 146.8 GB of SCSI storage internally.<br />
While there is no provision for external SCSI storage, there is an external<br />
Fibre Channel SAN storage option. Using the optional internal SCSI storage<br />
feature doubles the space requirement of the blade, cutting in half the number<br />
of blade servers that can be installed in a chassis.<br />
For large internal storage, the <strong>IBM</strong> Eserver xSeries® 345 is capable of<br />
holding up to 880.8 GB of hot-swap SCSI storage. All rack-mounted xSeries<br />
servers can support, either natively or via an optional ServeRAID adapter,<br />
external SCSI arrays.<br />
► Need for re-installation or repurposing at end of original: Stand-alone<br />
uniprocessor and 2-way servers can be distributed individually for use as<br />
departmental file/print servers and other low-horsepower uses. Because<br />
blade servers cannot be used without a chassis, the entire<br />
chassis-and-blades combo needs to stay together. <strong>The</strong> original chassis may<br />
be useful if older blade servers are discarded and newer, faster ones take<br />
their places in the chassis.<br />
► Power requirements: <strong>The</strong> data center is wired for 110-120 V power.<br />
<strong>BladeCenter</strong> requires 220-240 V power.<br />
Learn more about <strong>IBM</strong> Eserver <strong>BladeCenter</strong> and its components in the <strong>IBM</strong><br />
Redpaper <strong>The</strong> Cutting Edge: <strong>IBM</strong> Eserver <strong>BladeCenter</strong>, REDP-3581.<br />
<strong>The</strong> following chapters address the descriptions and considerations of the<br />
hardware and software components.<br />
Chapter 1. Introduction to <strong>BladeCenter</strong> and blade server technology 5
6 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Chapter 2. Hardware components<br />
2<br />
This chapter describes the major hardware components associated with the <strong>IBM</strong><br />
^ <strong>BladeCenter</strong> <strong>JS20</strong>. It begins by reviewing the core <strong>BladeCenter</strong><br />
infrastructure. <strong>The</strong>n it provides a detailed description of the <strong>BladeCenter</strong> <strong>JS20</strong><br />
and the range of options that you can order from <strong>IBM</strong>. Finally, this chapter<br />
reviews the <strong>IBM</strong> PowerPC® 970 RISC Microprocessor and <strong>IBM</strong> PowerPC 970FX<br />
RISC Microprocessor that are used as main processors on the <strong>BladeCenter</strong><br />
<strong>JS20</strong>.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 7
2.1 Overview of the <strong>BladeCenter</strong> infrastructure<br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> is designed to be installed in the <strong>IBM</strong> ^<br />
<strong>BladeCenter</strong> Type 8677. This section reviews the <strong>BladeCenter</strong> infrastructure that<br />
is relevant to the installation of the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
2.1.1 <strong>BladeCenter</strong> chassis<br />
<strong>The</strong> core component of the <strong>BladeCenter</strong> infrastructure is the <strong>BladeCenter</strong><br />
chassis. Figure 2-1 illustrates the front view of the <strong>BladeCenter</strong> chassis. <strong>The</strong><br />
<strong>BladeCenter</strong> chassis is designed to be installed in an industry standard 19-inch<br />
rack. Each <strong>BladeCenter</strong> chassis occupies seven rack units. You can install up to<br />
six <strong>BladeCenter</strong> chassis in a single 42U rack such as the <strong>IBM</strong> NetBAY42<br />
Enterprise Rack.<br />
In the front view of the <strong>BladeCenter</strong> chassis, you see these standard features:<br />
► Fourteen bays where you can install blade servers<br />
► A media bay containing one CD-ROM drive, one diskette drive, and a<br />
Universal Serial Bus (USB) port that can be dynamically assigned to any<br />
single blade server in the <strong>BladeCenter</strong> chassis<br />
Figure 2-1 <strong>BladeCenter</strong> chassis front view<br />
8 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Figure 2-2 illustrates the rear view of the <strong>BladeCenter</strong> chassis. In the rear of the<br />
<strong>BladeCenter</strong> chassis, you see these standard features:<br />
► One management module<br />
► One bay where you can install an optional redundant Management Module<br />
► Four bays where you can install optional input/output (I/O) modules<br />
► A redundant pair of power supply modules<br />
► Two bays where you can install an additional pair of redundant power supply<br />
modules<br />
► A redundant pair of blowers<br />
Figure 2-2 <strong>BladeCenter</strong> chassis rear view<br />
Chapter 2. Hardware components 9
Attention: Nonredundant power is not supported in <strong>BladeCenter</strong> products.<br />
Power modules must always be present in power bays 1 and 2. When any<br />
blade server or option is in blade bays 7 through 14, power modules must be<br />
present in all four power-module bays.<br />
If a power module fails or an ac power failure occurs, <strong>BladeCenter</strong> units<br />
configured for redundant power operation operate in a nonredundant mode, and<br />
the blower modules runs at full speed. You must replace the failing power module<br />
or restore ac power as soon as possible to regain redundant power operation and<br />
to reset the blower modules to their normal operating speed.<br />
As of the date of this publication, these four <strong>BladeCenter</strong> power-module options<br />
are supported:<br />
► <strong>IBM</strong> <strong>BladeCenter</strong> 1200W Power Supply Module (part number 48P7052)<br />
► <strong>IBM</strong> <strong>BladeCenter</strong> 1200W to 1400W Power Supply Upgrade Kit (part number<br />
90P0197)<br />
► <strong>IBM</strong> <strong>BladeCenter</strong> 1800W Power Supply Module (part number 13N0570)<br />
► <strong>IBM</strong> <strong>BladeCenter</strong> 2000W Power Supply Module (part number 26K4816)<br />
Not visible in either view of the <strong>BladeCenter</strong> chassis are the redundant pair of<br />
midplanes in the center of the <strong>BladeCenter</strong> chassis that interconnect the blade<br />
servers installed in the front, and other modules that are installed in the rear of<br />
the <strong>BladeCenter</strong> chassis. <strong>The</strong> midplanes support the hot-plugging of blade<br />
servers and the other <strong>BladeCenter</strong> modules.<br />
Note: <strong>IBM</strong> offers a variant of the <strong>BladeCenter</strong> called the <strong>BladeCenter</strong> T. <strong>The</strong><br />
<strong>BladeCenter</strong> T is designed to meet the special needs of the<br />
telecommunications industry and has been tested for NEBS compliance.<br />
Since <strong>BladeCenter</strong> <strong>JS20</strong> is not supported in the <strong>BladeCenter</strong> T at this time, we<br />
do not describe <strong>BladeCenter</strong> T and its associated features in this book.<br />
2.1.2 <strong>BladeCenter</strong> power options<br />
<strong>The</strong> standard redundant power supplies are installed in power bays 1 and 2 of<br />
the <strong>BladeCenter</strong> chassis, and provide power to the first six blade server bays.<br />
This is known as power domain A.<br />
To install blade servers in the remaining bays 7 through 14, you must install an<br />
additional pair of redundant power supply modules in power bays 3 and 4. This is<br />
known as power domain B. Figure 2-3 shows these two power domains.<br />
10 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Figure 2-3 <strong>BladeCenter</strong> power domains<br />
Chapter 2. Hardware components 11
Attention: Power supply modules must always be installed in pairs. You can<br />
have either two power supply modules (in power bays 1 and 2) or four power<br />
supply modules (in power bays 1 through 4). Both power supply modules in a<br />
redundant pair should have the same capacity.<br />
When setting up the power requirements for the <strong>BladeCenter</strong>, remember the<br />
following criteria:<br />
► Install the power modules in pairs in a domain. <strong>The</strong>y must match each other in<br />
capacity (wattage, amperage, and so on).<br />
► A power domain operating above the capacity of a single power module<br />
results in a nonredundant power condition.<br />
► In a pair of power modules, a power module that is not connected to a<br />
200–240 volt ac power source results in a nonredundant power condition.<br />
► To provide true redundant power, connect <strong>BladeCenter</strong> power modules 1<br />
and 3 to a different 200–240 volt ac power source than to power modules 2<br />
and 4.<br />
► Connect an installed power module to an ac power source. <strong>The</strong> module must<br />
not be used as a filler.<br />
Attention: To maintain proper system cooling, each unoccupied blade bay<br />
must contain a filler blade as well as each unoccupied I/O,<br />
management-module, or power-module bay. An installed power module must<br />
be powered, but must not be used as a filler.<br />
Table 2-1 shows the minimum power requirements for power domain A.<br />
Table 2-1 Power domain A minimum power requirements<br />
Sum of <strong>JS20</strong> power units Minimum power module required<br />
Less than 7.4 1200 W (labeled 7.5A)<br />
7.4 up to less than 9.0 1400 W (labeled 9A)<br />
9.0 up to less than 10.0 1800 W (labeled 12A)<br />
10.0 or greater 2000 W (labeled 13.5A)<br />
12 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Table 2-2 shows the minimum power requirements for power domain B.<br />
Table 2-2 Power domain B minimum power requirements<br />
Sum of <strong>JS20</strong> power units Minimum power module required<br />
Less than 9.9 1200 W (labeled 7.5A)<br />
9.9 up to less than 11.5 1400 W (labeled 9A)<br />
11.5 up to less than 13.4 1800 W (labeled 12A)<br />
13.4 or greater 2000 W (labeled 13.5A)<br />
Each <strong>BladeCenter</strong> <strong>JS20</strong> (1.6 GHz/800 MHz 8842-21X) requires 1.5 power units<br />
for two installed microprocessors. For example, if you fully populate all six bays,<br />
the total power units is 9.0 (6 bays times 1.5). According to Table 2-1, you need a<br />
minimum of the 1800 W power module.<br />
For other configurations, refer to the latest version of <strong>IBM</strong> ^ <strong>BladeCenter</strong><br />
Power Module Upgrade Guidelines Technical Update, found at:<br />
http://www.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-53353<br />
Also, you can use the following steps to obtain the <strong>BladeCenter</strong> Planning and<br />
Installation Guide:<br />
1. Go to:<br />
http://www.ibm.com/pc/support/<br />
2. In the Browse by product section, click Servers, and then select the brand<br />
Servers and the family <strong>BladeCenter</strong>.<br />
3. Click Continue.<br />
4. From the View by document type menu, click Online Publications.<br />
2.1.3 Management Module<br />
Each <strong>BladeCenter</strong> chassis comes standard with a Management Module, as<br />
shown in Figure 2-4. You can also install a redundant Management Module by<br />
ordering the <strong>IBM</strong> KVM/Redundant Management Module Option (P/N 48P7055).<br />
When you have two Management Modules, only one is active. <strong>The</strong> other module<br />
is in standby mode until it is switched to become the active Management Module.<br />
Chapter 2. Hardware components 13
Figure 2-4 KVM Management Module<br />
<strong>The</strong> Management Module provides the following three major functions:<br />
► Console switching and remote graphical console for blade servers that use a<br />
keyboard, video monitor, and mouse (KVM) for their console<br />
► Serial over local area network (LAN) (SoL) remote text consoles for blade<br />
servers that support SoL consoles (includes the <strong>BladeCenter</strong> <strong>JS20</strong>)<br />
► Service processor for the entire <strong>BladeCenter</strong><br />
Important: <strong>The</strong> console switching and remote graphical console functionality<br />
of the Management Module is not used by the <strong>BladeCenter</strong> <strong>JS20</strong> because it<br />
does not support KVM consoles. You can still use the console switching and<br />
remote graphical console functions if you have other blade servers installed in<br />
the same <strong>BladeCenter</strong> chassis that support KVM consoles.<br />
You use the service processor functionality of the Management Module to<br />
perform many different tasks to:<br />
► Initially configure <strong>BladeCenter</strong><br />
► Define and manage the users who can remotely access the Management<br />
Module<br />
► Monitor the <strong>BladeCenter</strong> and reporting abnormal conditions to system<br />
managers either via e-mail or Simple Network Management Protocol (SNMP)<br />
events sent to management systems<br />
► Remotely power on and off individual blade servers and I/O modules<br />
14 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
► Remotely switch the CD-ROM, diskette, and USB interfaces in the media bay<br />
of the <strong>BladeCenter</strong> chassis to a particular blade server<br />
► Maintain the firmware embedded in various <strong>BladeCenter</strong> components<br />
<strong>The</strong> Management Module is connected to each blade server and I/O module via<br />
management buses embedded in the <strong>BladeCenter</strong> chassis midplanes. <strong>The</strong><br />
Management Module uses these connections to provide service processor<br />
functionality in conjunction with the Blade Systems Management Processor<br />
(BSMP) on each blade server.<br />
When using the SoL remote text console function, the Management Module also<br />
uses a virtual LAN (VLAN) provided by the LAN switch module installed in I/O<br />
module bay 1 to communicate with each blade server. This VLAN is not<br />
accessible externally.<br />
Each Management Module is equipped with several connectors, as illustrated in<br />
Figure 2-5.<br />
Figure 2-5 Management Module connectors<br />
Chapter 2. Hardware components 15
2.1.4 I/O modules<br />
You use the keyboard, video, and mouse connectors to attach a local KVM<br />
console that can be switched to any blade server that supports KVM consoles.<br />
You use the 10/100BaseT Ethernet connector to attach the Management Module<br />
to an external Ethernet network. You use this interface to access the remote<br />
graphical console, SoL consoles, and the service processor functionality of the<br />
Management Module.<br />
You can access the various functions of the Management Module via the<br />
Ethernet interface using several different methods:<br />
► Using a graphical interface through a Web browser (HTTP or HTTPS)<br />
► Using a command line interface through a Telnet or SSH client<br />
► Using system management software such as <strong>IBM</strong> Director or <strong>IBM</strong> Cluster<br />
Systems Management<br />
We demonstrate these methods in Chapter 5, “Hardware setup” on page 69, and<br />
in Chapter 8, “System management scenarios” on page 139.<br />
I/O modules enable connectivity between blade servers within the same chassis,<br />
and between blade servers and the outside world. You can install up to four I/O<br />
modules in a <strong>BladeCenter</strong> chassis, one in each I/O module bay.<br />
<strong>The</strong> available I/O modules can be used to provide connectivity to three different<br />
types of network:<br />
► Local area networks<br />
► Storage area networks (SANs)<br />
► High-bandwidth low-latency networks used in clustering (Myrinet)<br />
Most I/O modules support connectivity to only one type of network. <strong>The</strong><br />
exception is the pass-through module, which can be used with all three types of<br />
networks.<br />
Each I/O module is connected to every blade server via the <strong>BladeCenter</strong> chassis<br />
midplanes. Each single width blade server can have up to four interfaces, one to<br />
each I/O module bay. Double-width blade servers can have up to eight interfaces,<br />
two to each I/O module bay.<br />
Each blade server must have a compatible interface to communicate with an I/O<br />
module. Every blade server that is currently available has standard Gigabit<br />
Ethernet interfaces that are connected to I/O module bays 1 and 2. <strong>The</strong>refore,<br />
you can use only a LAN Switch or Pass-Thru I/O Module in these I/O module<br />
bays.<br />
16 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Restriction: You must install one of the LAN switch modules in I/O module<br />
bay 1 when using the SoL remote text console function. This function depends<br />
on a private VLAN provided by the LAN switch to function correctly.<br />
Every single-width blade server currently available supports the installation of an<br />
optional expansion card that connects to I/O module bay 3 and 4. Double-width<br />
blade servers support two optional expansion cards. If you install an expansion<br />
card, it must be compatible with any I/O modules installed in I/O module bays<br />
3 and 4.<br />
You can order the following LAN Switch I/O Modules:<br />
► P/N 13N0568: 4 port Gigabit Ethernet Switch Module<br />
► P/N 13N2281: CISCO Systems Intelligent Gigabit Ethernet Switch Module<br />
► P/N 73P9057: Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module<br />
You can order the following SAN switch I/O modules:<br />
► P/N 48P7062: 2-Port Fibre Channel Switch Module<br />
► P/N 26K5601: Brocade Entry SAN Switch Module<br />
► P/N 90P0165: Brocade Enterprise SAN Switch Module<br />
For the Pass-Thru I/O Module, you can order Optical Pass-thru Module (P/N<br />
02R9080).<br />
We describe each of these modules in more detail in the sections that follow.<br />
<strong>IBM</strong> 4-Port Gigabit Ethernet Switch Module<br />
<strong>The</strong> <strong>IBM</strong> 4-Port Gigabit Ethernet Switch Module is a layer 2 LAN switch that<br />
provides four external 10/100/1000BaseT interfaces for connecting to LANs that<br />
are external to the <strong>BladeCenter</strong> chassis. Internally the switch module is designed<br />
to connect to a single Gigabit Ethernet interface on each installed single-width<br />
blade server, or two such interfaces on each installed double-width blade server.<br />
<strong>The</strong> <strong>IBM</strong> 4-Port Gigabit Ethernet Switch Module is also connected via a Fast<br />
Ethernet interface to each installed Management Module. This is used to<br />
remotely access the switch module via the Management Module’s 10/100BaseT<br />
Ethernet interface. You can use this capability to support initial configuration and<br />
management of the switch module.<br />
Major features of the 4-Port Gigabit Ethernet Switch Module include:<br />
► VLAN support based on either port or 802.1Q tagging<br />
► Link aggregation of multiple external interfaces to provide increased<br />
bandwidth to external networks<br />
Chapter 2. Hardware components 17
Note: <strong>The</strong> <strong>IBM</strong> 4-Port Gb Ethernet Switch Module for <strong>BladeCenter</strong> may be<br />
installed in your <strong>BladeCenter</strong> unit and you must replace the CD-ROM media<br />
tray in your <strong>BladeCenter</strong> unit. In this case, you must also update the<br />
microcode in the Ethernet switch module.<br />
CISCO Systems Intelligent Gigabit Ethernet Switch Module<br />
<strong>The</strong> CISCO Systems Intelligent Gigabit Ethernet Switch Module (IGESM) is a<br />
layer 2-3 LAN switch that provides four external 10/100/1000BaseT interfaces for<br />
connecting to LANs external to the <strong>BladeCenter</strong> chassis. Internally the switch<br />
module is designed to connect to a single Gigabit Ethernet interface on each<br />
installed single width blade server, or two such interfaces on each installed<br />
double-width blade server.<br />
<strong>The</strong> IGESM is also connected via a Fast Ethernet interface to each installed<br />
Management Module. This is used to remotely access the switch module via the<br />
Management Module’s 10/100BaseT Ethernet interface. You can use this<br />
capability to support initial configuration and management of the switch module.<br />
<strong>The</strong> IGESM uses CISCO IOS. Use the IGESM when you require the advanced<br />
functionality of CISCO IOS or maximum compatibility with external networks that<br />
constructed using switches running CISCO IOS.<br />
Note: To use the SoL remote text console function with the IGESM, you must<br />
manually create the required VLAN using the procedures in the CISCO<br />
Systems Intelligent Gigabit Ethernet Switch Module - Installation Guide.<br />
Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module<br />
<strong>The</strong> Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module is a layer 2-7<br />
LAN switch that provides four external 10/100/1000BaseT interfaces for<br />
connecting to LANs external to the <strong>BladeCenter</strong> chassis. Internally the switch<br />
module is designed to connect to a single Gigabit Ethernet interface on each<br />
installed single width blade server, or two such interfaces on each installed<br />
double-width blade server.<br />
<strong>The</strong> Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module is also<br />
connected via a Fast Ethernet interface to each installed Management Module.<br />
This is used to remotely access the switch module via the Management Module’s<br />
10/100BaseT Ethernet interface. You can use this capability to support initial<br />
configuration and management of the switch module.<br />
18 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
<strong>The</strong> Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module provides several<br />
advanced networking features including:<br />
► Rich layer 2 and layer 3 switching capabilities<br />
► Virtual server load balancing (layer 4 switching)<br />
► Content aware load balancing (layer 7 switching)<br />
► High availability clustering of switch modules in the same or different chassis<br />
You can learn more about the Nortel Networks Layer 2-7 Gigabit Ethernet Switch<br />
Module in the <strong>IBM</strong> Redpaper <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Layer 2-7 Network<br />
Switching, REDP-3755.<br />
2-Port Fibre Channel Switch Module<br />
<strong>The</strong> 2-Port Fibre Channel Switch Module is a Fibre Channel SAN switch that<br />
provides two external 2 Gbps Fibre Channel interfaces for connecting to SANs<br />
external to the <strong>BladeCenter</strong> chassis. Internally the switch module is designed to<br />
connect to a single 2 Gbps Fibre Channel interface on each installed blade<br />
server that is equipped with a Fibre Channel expansion card.<br />
<strong>The</strong> 2-Port Fibre Channel Switch Module is also connected via a Fast Ethernet<br />
interface to each installed Management Module. This is used to remotely access<br />
the switch module via the Management Module’s 10/100BaseT Ethernet<br />
interface. You can use this capability to support initial configuration and<br />
management of the switch module.<br />
<strong>The</strong> 2-Port Fibre Channel Switch Module can be installed only in I/O module<br />
bays 3 or 4. If you install a 2-Port Fibre Channel Switch Module, any blade<br />
servers that have expansion cards must use a Fibre Channel expansion card.<br />
Brocade Enterprise SAN Switch Module<br />
<strong>The</strong> Brocade Enterprise SAN Switch Module is a Fibre Channel SAN switch that<br />
provides two external 1 Gbps or 2 Gbps (autosense) Fibre Channel interfaces for<br />
connecting to SANs external to the <strong>BladeCenter</strong> chassis. Internally the switch<br />
module is designed to connect to a single 2 Gbps Fibre Channel interface on<br />
each installed blade server that is equipped with a Fibre Channel expansion<br />
card.<br />
<strong>The</strong> Brocade Enterprise SAN Switch Module is also connected via a Fast<br />
Ethernet interface to each installed Management Module. This is used to<br />
remotely access the switch module via the Management Module’s 10/100BaseT<br />
Ethernet interface. You can use this capability to support initial configuration and<br />
management of the switch module.<br />
Chapter 2. Hardware components 19
<strong>The</strong> Brocade Enterprise SAN Module can be installed only in I/O module bays<br />
3 or 4. If you install a 2-Port Fibre Channel Switch Module, any blade servers that<br />
have expansion cards must use a Fibre Channel expansion card.<br />
Use the Brocade Enterprise SAN Switch Module when you require maximum<br />
compatibility with external SAN fabrics constructed using Brocade-based<br />
switches. <strong>The</strong> Brocade Enterprise SAN Switch Module is functionality equivalent<br />
to the Brocade Silkworm 3900 SAN switch.<br />
Brocade Entry SAN Switch Module<br />
<strong>The</strong> Brocade Entry SAN Switch Module is almost identical to the Brocade<br />
Enterprise SAN Switch Module described earlier. However, it is limited to<br />
participating in SAN fabrics that contain at most two SAN switches.<br />
Optical Pass-thru Module<br />
<strong>The</strong> Optical Pass-thru Module provides the capability to access one of the<br />
network interfaces on each blade server via an externally accessible optical<br />
interface. You can use this optical interface to connect the blade servers to<br />
external switches that support compatible optical interfaces, including:<br />
► LAN switches<br />
► SAN switches<br />
► Myrinet switches<br />
Each Optical Pass-thru Module has four multi-port optical connectors. <strong>The</strong><br />
connectors are designed for special break-out cables that provide either four SC<br />
connectors (P/N 73P5992) or four LC connectors (P/N 73P6033) per multi-port<br />
optical connector. You can connect to a maximum of 14 blade server network<br />
interfaces using four of these cables (two of the connectors are unused on the<br />
fourth cable).<br />
<strong>The</strong> type of external switch that you connect to the Optical Pass-thru Module<br />
must be compatible with the network interface on the blade servers that are<br />
connected to the I/O module bay where the Optical Pass-thru Module is installed.<br />
If you install an Optical Pass-thru Module in I/O module bay 1 or 2, you must<br />
connect the optical interfaces to a Gigabit Ethernet switch with 1000Base-SX<br />
interfaces.<br />
Restriction: When a <strong>BladeCenter</strong> chassis contains a <strong>BladeCenter</strong> <strong>JS20</strong>,<br />
always use a LAN switch module in I/O module bay 1 to support the SoL<br />
remote text console function. This is because there is no other mechanism for<br />
providing a console on a <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
20 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
2.1.5 Blade servers<br />
If you install an Optical Pass-thru Module in I/O module bay 3 or 4, you must<br />
connect the optical interfaces to an external switch that is compatible with the<br />
type of expansion cards installed on blade servers.<br />
If you install a Myrinet expansion card on any blade server, you must install an<br />
Optical Pass-thru Module in I/O module bay 4 to enable the Myrinet interface on<br />
the expansion card to be connected to an external Myrinet switch. This is<br />
because currently no Myrinet switch modules are available for the <strong>BladeCenter</strong>.<br />
<strong>IBM</strong> offers a variety of blade servers that you can install in a <strong>BladeCenter</strong><br />
chassis.<br />
► <strong>IBM</strong> ^ <strong>BladeCenter</strong> <strong>JS20</strong> Type 8842 Blade Server<br />
► <strong>IBM</strong> ^ <strong>BladeCenter</strong> HS20 Type 8678 Blade Server<br />
► <strong>IBM</strong> ^ <strong>BladeCenter</strong> HS20 Type 8832 Blade Server<br />
► <strong>IBM</strong> ^ <strong>BladeCenter</strong> HS40 Type 8839 Blade Server<br />
Because the <strong>BladeCenter</strong> <strong>JS20</strong> is the subject of this book, we describe it in<br />
greater detail in 2.2, “<strong>BladeCenter</strong> <strong>JS20</strong>” on page 22.<br />
<strong>The</strong> following sections briefly introduce the other blade servers. You can install<br />
any combination of blade servers in the same <strong>BladeCenter</strong> chassis subject to<br />
available power supply module capacity.<br />
HS20 Blade Server<br />
<strong>The</strong> HS20 Blade Server is a single-width blade server that contains either one or<br />
two Intel® Xeon DP processors and up to 8 GB of memory. Up to 14 HS20 Blade<br />
Servers can be installed in each <strong>BladeCenter</strong> chassis.<br />
<strong>The</strong> original HS20 Type 8678 Blade Server used a 400 MHz front side bus (FSB)<br />
and supports a range of processor speeds from 2.0 GHz to 2.8 GHz. <strong>The</strong> HS20<br />
Type 8678 Blade Server is now withdrawn from marketing and has been replaced<br />
by the HS20 Type 8832 Blade Server.<br />
<strong>The</strong> HS20 Type 8832 Blade Server uses a 533 MHz FSB and supports a range<br />
of processor speeds from 2.4 Ghz to 3.2 GHz.<br />
HS40 Blade Server<br />
<strong>The</strong> HS40 Blade Server is a double-width blade server that contains from one to<br />
four Intel Xeon MP processors and up to 16 GB of memory. Up to seven HS40<br />
Blade Servers can be installed in each <strong>BladeCenter</strong> chassis.<br />
Chapter 2. Hardware components 21
2.2 <strong>BladeCenter</strong> <strong>JS20</strong><br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> is a single-width blade server that offers these features:<br />
► Two PowerPC 970 or PowerPC 970FX microprocessors<br />
► Up to 4 GB of RAM<br />
► Up to two integrated development environment (IDE) hard disk drives<br />
mounted on the blade server<br />
► Two integrated Gigabit Ethernet interfaces<br />
► Expansion option connectors for an expansion card option to provide either:<br />
– Two additional Gigabit Ethernet interfaces<br />
– Two 2 Gbps Fibre Channel interfaces<br />
– One Myrinet interface<br />
► Blade Systems Management Processor<br />
Figure 2-6 illustrates the physical layout of the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
I/O expansion option<br />
I/O expansion option<br />
connector (J18)<br />
connector (J22)<br />
Four Memory DIMMS<br />
Secondary IDE (J2) Primary IDE (J1) Battery<br />
(shown with attached disk)<br />
Figure 2-6 <strong>BladeCenter</strong> <strong>JS20</strong> physical layout<br />
22 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
Microprocessor 0<br />
and heat sink (U87)<br />
Microprocessor 1<br />
and heat sink (U41)
<strong>The</strong> architecture of the <strong>BladeCenter</strong> <strong>JS20</strong> is illustrated by the diagram in<br />
Figure 2-7.<br />
PowerPC<br />
970<br />
or<br />
970FX<br />
6.4 GBps<br />
or<br />
8.8 GBps<br />
PowerPC<br />
970<br />
or<br />
970FX<br />
Memory Controller and Host I/O Bridge<br />
(Northbridge)<br />
HyperTransport PCI-X Tunnel<br />
HyperTransport I/O Hub<br />
IDE Controller<br />
IDE<br />
Disk<br />
IDE<br />
Disk<br />
1.6 GBps<br />
800 MBps<br />
6.4 GBps<br />
or<br />
8.8 GBps<br />
5.3 GBps<br />
Real Time<br />
Clock<br />
1.06 GBps<br />
1.06 GBps<br />
USB<br />
Controllers<br />
Super I/O<br />
Figure 2-7 <strong>BladeCenter</strong> <strong>JS20</strong> block diagram<br />
UART<br />
BSMP<br />
PC2700 ECC<br />
Memory Modules<br />
Gigabit<br />
Ethernet<br />
Controller<br />
Expansion Card<br />
Boot<br />
FLASH<br />
NVRAM<br />
Chassis<br />
Midplanes<br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> includes both base features and a range of optional<br />
features. We describe each of these in the sections that follow.<br />
Chapter 2. Hardware components 23
2.2.1 Base features<br />
<strong>The</strong>re are two models of the <strong>BladeCenter</strong> <strong>JS20</strong>. Both models have the same<br />
base features with the exception of the microprocessors used in the processor<br />
subsystem. <strong>The</strong> <strong>JS20</strong> Type 8842-21X Blade Server uses the PowerPC 970<br />
microprocessor. <strong>The</strong> <strong>JS20</strong> Type 8842-41X Blade Server uses the PowerPC<br />
970FX microprocessor.<br />
Processor subsystem<br />
<strong>The</strong> processor subsystem of the <strong>BladeCenter</strong> <strong>JS20</strong> is comprised of either two<br />
PowerPC 970 or two PowerPC 970FX microprocessors. <strong>The</strong>y operate at internal<br />
clock frequencies of either 1.6 GHz or 2.2 GHz, respectively.<br />
Each processor includes a 32 KB level 1 data cache, a 64 KB level 1 instruction<br />
cache, and a 512 KB ECC level 2 cache.<br />
Each processor is also interconnected with the rest of the system via a dedicated<br />
interface to the memory controller and host I/O bridge (Northbridge). <strong>The</strong><br />
interface to each processor supports a maximum aggregate bandwidth of<br />
6.4 GBps (for processors operating at 1.6 GHz) or 8.8 GBps (for processors<br />
operating at 2.2 GHz).<br />
You can learn more about the features and architecture of the processors in 2.3,<br />
“PowerPC 970 and PowerPC 970FX Microprocessors” on page 29.<br />
Memory subsystem<br />
<strong>The</strong> memory subsystem is comprised of the memory controller and up to four<br />
DDR1 333 MHz (PC2700) ECC memory modules. <strong>The</strong> memory controller uses<br />
two-way interleaving to increase the available peak memory bandwidth to 5.3<br />
GBps (333 MHz x 16 bytes).<br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> includes two 256 MB memory modules that provide a<br />
total memory capacity of 512 MB. <strong>The</strong>se memory modules are installed in slots 3<br />
and 4. <strong>The</strong> remaining memory module slots (1 and 2) are empty in the base<br />
<strong>BladeCenter</strong> <strong>JS20</strong>.<br />
You can expand the available memory by installing optional memory modules.<br />
<strong>The</strong> available memory modules are described in “Memory modules” on page 27.<br />
I/O subsystem<br />
<strong>The</strong> I/O subsystem is connected to the processor and memory subsystems via a<br />
hyper transport link that supports an aggregate bandwidth of 1.6 GBps.<br />
Downstream of this link is a range of I/O devices and expansion interfaces as<br />
explained in the following sections.<br />
24 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Gigabit Ethernet controller<br />
Each <strong>BladeCenter</strong> <strong>JS20</strong> includes a Broadcom BCM5704s Gigabit Ethernet<br />
controller, which provides the primary mechanism for connecting the<br />
<strong>BladeCenter</strong> <strong>JS20</strong> to other systems. This controller provides two independent<br />
Gigabit Ethernet interfaces.<br />
Each Gigabit Ethernet interface is connected to a separate <strong>BladeCenter</strong> I/O<br />
module bay via the <strong>BladeCenter</strong> midplanes. <strong>The</strong> first interface is connected to I/O<br />
module bay 1, and the second interface is connected to I/O module bay 2.<br />
To use a Gigabit Ethernet interface, install a compatible I/O module in the<br />
corresponding I/O module bay. Install a LAN Switch I/O Module in I/O module<br />
bay 1 to support remote access to the <strong>BladeCenter</strong> <strong>JS20</strong> text console via SoL.<br />
I/O expansion option connectors<br />
Each <strong>BladeCenter</strong> <strong>JS20</strong> contains I/O expansion option connectors, which you<br />
can use to attach one of several different expansion cards. This capability<br />
enables you to expand the external I/O connectivity of the <strong>BladeCenter</strong> <strong>JS20</strong> to<br />
include Fibre Channel SANs, Myrinet high-bandwidth low-latency networks, and<br />
additional Gigabit Ethernet LAN interfaces.<br />
<strong>The</strong> expansion card installed in the I/O expansion option connectors can access<br />
I/O modules installed in <strong>BladeCenter</strong> I/O module slots 3 and 4 via the<br />
<strong>BladeCenter</strong> midplanes.<br />
Note: <strong>The</strong> main I/O expansion option connector provides a PCI-X electrical<br />
interface. However, a special form factor is required to enable installation<br />
within the confined dimensions of a blade server. <strong>The</strong>refore, standard PCI-X<br />
adapter cards cannot be used with the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
You can learn more about these expansion cards in “Expansion cards” on<br />
page 28.<br />
IDE controller<br />
Each <strong>BladeCenter</strong> <strong>JS20</strong> includes a dual-channel IDE controller and two<br />
connectors for 2.5-inch IDE hard disk drives. You can use this capability to install<br />
one or two optional 2.5-inch IDE hard disk drives on the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
Note: You must install at least one hard disk drive for use as a boot device<br />
unless you plan to install a Fibre Channel Expansion Card and configure the<br />
<strong>BladeCenter</strong> <strong>JS20</strong> to boot from a SAN. You can also boot from a network<br />
interface to support the network installation of an operating system.<br />
Chapter 2. Hardware components 25
You can learn more about the available hard disk drives in “Expansion cards” on<br />
page 28.<br />
USB controllers<br />
Each <strong>BladeCenter</strong> <strong>JS20</strong> includes two USB 1.1 host controllers. <strong>The</strong>y are<br />
connected via the <strong>BladeCenter</strong> midplanes to the USB peripherals located in the<br />
media bay of the <strong>BladeCenter</strong> chassis.<br />
You use these controllers when accessing either the CD-ROM or diskette drive<br />
that are located in the media bay of the <strong>BladeCenter</strong> chassis. You can also<br />
access other USB devices that may be attached to the USB connector located in<br />
the media bay of the <strong>BladeCenter</strong> chassis.<br />
Note: Verify that the USB device is supported by the operating system<br />
installed on the blade server before you attach it to the USB connector located<br />
in the media bay of the <strong>BladeCenter</strong> chassis.<br />
You can assign only the USB peripherals to one blade server at a time. Either<br />
use a button on the front of the blade server or one of the remote interfaces to the<br />
Management Module to switch the USB peripherals to a specific blade server.<br />
Miscellaneous I/O<br />
Each <strong>BladeCenter</strong> <strong>JS20</strong> incorporates various miscellaneous I/O devices that are<br />
necessary to implement a functional server. <strong>The</strong>se are illustrated in the<br />
<strong>BladeCenter</strong> <strong>JS20</strong> diagram in Figure 2-7 on page 23 and include:<br />
► A real-time clock (RTC) for maintaining the date and time<br />
► Non-volatile memory (NVRAM) for storing configuration settings<br />
► Boot FLASH memory for storing the firmware (BIOS) for the blade server<br />
► A serial port (UART)<br />
<strong>The</strong> serial port is connected within the blade server to the BSMP. <strong>The</strong> BSMP<br />
participates in the implementation of the SoL remote text console function that<br />
provides remote connectivity to the serial port via the <strong>BladeCenter</strong> Management<br />
Module.<br />
Blade Systems Management Processor<br />
Each <strong>BladeCenter</strong> <strong>JS20</strong> has a BSMP. <strong>The</strong> BSMP provides the interface that<br />
enables the <strong>BladeCenter</strong> Management Module to manage the <strong>BladeCenter</strong> <strong>JS20</strong><br />
via the <strong>BladeCenter</strong> chassis midplanes.<br />
<strong>The</strong> BSMP is responsible for powering up the <strong>BladeCenter</strong> <strong>JS20</strong>, configuring the<br />
various hardware subsystems, monitoring temperatures, and driving the status<br />
indicators on the blade server front panel.<br />
26 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
<strong>The</strong> BSMP also participates in the implementation of the SoL remote text<br />
console function on the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
2.2.2 Optional features<br />
You can install several optional features on a <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
Memory modules<br />
Each <strong>BladeCenter</strong> <strong>JS20</strong> is equipped with 512 MB of memory. This is provided<br />
using a pair of 256 MB memory modules.<br />
You can also install the following optional memory modules on a <strong>BladeCenter</strong><br />
<strong>JS20</strong>:<br />
► P/N 73P2275: <strong>IBM</strong> 512 MB 333 MHz PC2700 CL2.5 ECC DDR RDIMM<br />
► P/N 73P2276: <strong>IBM</strong> 1 GB 333 MHz PC2700 CL2.5 ECC DDR RDIMM<br />
You must install the optional memory module features in identical pairs. This is<br />
necessary because the memory controller on the <strong>BladeCenter</strong> <strong>JS20</strong> uses<br />
two-way interleaving.<br />
Two spare memory module slots (1 and 2) are available on the <strong>BladeCenter</strong><br />
<strong>JS20</strong>. You can expand the available memory to either 1.5 GB or 2.5 GB, using<br />
either a pair of 512 MB or 1 GB memory modules respectively.<br />
You can also remove the standard pair of 256 MB memory modules and replace<br />
them by a pair of either 512 MB or 1 GB memory modules if further memory<br />
expansion is required. To obtain the maximum supported memory configuration<br />
of 4 GB, remove both of the base 256 MB memory modules and install four 1 GB<br />
memory module features.<br />
Hard disk drives<br />
<strong>The</strong> base <strong>BladeCenter</strong> <strong>JS20</strong> is not equipped with any hard disk drives. You can<br />
install one or two of the 2.5-inch IDE hard disk drive option 40 GB 5400 rpm<br />
ATA-100 Hard Disk Drive (P/N 48P7063).<br />
When you install two hard disk drives, each hard disk drive is connected to its<br />
own dedicated IDE bus.<br />
Restriction: If you install an expansion card on a <strong>BladeCenter</strong> <strong>JS20</strong>, only one<br />
hard disk drive can be installed on the same blade server.<br />
You do not have to install a hard disk drive if you have installed a Fibre Channel<br />
Expansion Card and have configured the <strong>BladeCenter</strong> <strong>JS20</strong> to boot from a SAN.<br />
Chapter 2. Hardware components 27
Expansion cards<br />
You can install any one of the following expansion cards in the I/O expansion<br />
option connectors of the <strong>BladeCenter</strong> <strong>JS20</strong>:<br />
► P/N 73P9030: Gigabit Ethernet Expansion Card<br />
► P/N 13N2203: Fibre Channel Expansion Card<br />
► P/N 73P6000: Myrinet Cluster Expansion Card<br />
We describe each of these expansion cards in the sections that follow.<br />
Fibre Channel Expansion Card<br />
You can use the Fibre Channel Expansion Card to connect the <strong>BladeCenter</strong><br />
<strong>JS20</strong> to SANs. <strong>The</strong> Fibre Channel Expansion Card provides two 2 Gbps Fibre<br />
Channel interfaces that are connected to I/O module bays 3 and 4. You must<br />
install either a SAN switch I/O module or the Optical Pass-Thru I/O Module in one<br />
or both of these I/O module bays to connect the blade server Fibre Channel<br />
interfaces to an external SAN.<br />
Note: <strong>The</strong>re have been two different versions of the Fibre Channel Expansion<br />
Card. Both versions are functionally equivalent. <strong>The</strong> earlier version (P/N<br />
48P7061) is now withdrawn from marketing. However, you may still find it in<br />
existing <strong>BladeCenter</strong> installations.<br />
Gigabit Ethernet Expansion Card<br />
You can use the Gigabit Ethernet Expansion Card to increase the number of<br />
Gigabit Ethernet network interfaces on the <strong>BladeCenter</strong> <strong>JS20</strong> from two to four.<br />
<strong>The</strong> Gigabit Ethernet Expansion Card provides two Gigabit Ethernet interfaces<br />
that are connected to I/O module bays 3 and 4. You must install either a LAN<br />
Switch I/O Module or the Optical Pass-Thru I/O Module in one or both of these<br />
I/O module bays to connect the Gigabit Ethernet interfaces on the expansion<br />
card to an external LAN.<br />
Myrinet Cluster Expansion Card<br />
You can use the Myrinet Cluster Expansion Card to connect the <strong>BladeCenter</strong><br />
<strong>JS20</strong> to a Myrinet network. Myrinet was developed by Myricom Inc., and<br />
conforms to an ANSI standard (ANSI/VITA 26-1998).<br />
Myrinet networks are typically used to support certain types of high performance<br />
computing applications that distribute computation across a cluster of multiple<br />
servers. Such applications are said to exploit distributed memory parallelism.<br />
Distributed memory parallel applications often require the servers in the cluster<br />
to exchange messages with each other through some form of interconnection<br />
28 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
network. <strong>The</strong> performance of such applications often depends on how quickly<br />
these messages can be exchanged. <strong>The</strong> highest performance is usually<br />
obtained by using a high-bandwidth, low-latency interconnection network.<br />
Myrinet provides a high-bandwidth, low-latency interconnection network that can<br />
be used to optimize the performance of distributed memory parallel applications.<br />
Consider using Myrinet if your application requires higher performance than is<br />
achievable using Gigabit Ethernet technology.<br />
You can learn more about Myrinet technology at the Myrinet Web site at:<br />
http://www.myri.com<br />
Note: <strong>The</strong> Myrinet Cluster Expansion Card is a repackaging of the Myricom<br />
M3F-PCIXD-2 PCI-X adapter to fit the special form factor required for blade<br />
server expansion cards.<br />
When you install a Myrinet Cluster Expansion Card on a blade server, you must<br />
install an Optical Pass-thru Module in I/O module bay 4 of the <strong>BladeCenter</strong><br />
chassis. You must also connect the external interfaces of the Optical Pass-thru<br />
Module to an external Myrinet switch via appropriate optical fiber cables.<br />
We elaborate on the planning implications associated with using the Myrinet<br />
Cluster Expansion Card in 4.1.2, “High-performance, low-latency network<br />
requirements” on page 58.<br />
You can order Myrinet switches from <strong>IBM</strong> as a component of the <strong>IBM</strong> ^<br />
Cluster 1350 product. You can learn more about Cluster 1350 at:<br />
http://www.ibm.com/eserver/cluster<br />
2.3 PowerPC 970 and PowerPC 970FX Microprocessors<br />
This section provides more detailed information about the <strong>IBM</strong> PowerPC 970<br />
Microprocessor and <strong>IBM</strong> PowerPC 970FX RISC Microprocessor that are used for<br />
the main processors on the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
This section reviews the POWER and PowerPC Architecture upon which the<br />
PowerPC 970 and PowerPC 970FX microprocessors are based. It also<br />
introduces the Vector/SIMD Multimedia eXtension (VMX) to the PowerPC<br />
Architecture. And it describes the major features of the PowerPC 970 and<br />
PowerPC 970FX microprocessors.<br />
Chapter 2. Hardware components 29
2.3.1 Review of POWER and PowerPC Architecture<br />
In 1990, <strong>IBM</strong> announced a new processor architecture called performance<br />
optimization with enhanced RISC (POWER). This architecture was first<br />
implemented in the <strong>IBM</strong> RS/6000®.<br />
<strong>The</strong> POWER architecture was based on earlier <strong>IBM</strong> Research efforts to develop<br />
reduced instruction set computer (RISC) technology. <strong>The</strong>se efforts began in the<br />
1970s and continued throughout the 1980s.<br />
RISC and CISC concepts<br />
In the 1960s and 1970s, most computers were of a design now known as a<br />
complex instruction set computer (CISC). This type of design featured a large<br />
and highly functional instruction set. Each instruction in a CISC processor design<br />
typically needs several processor clock cycles to execute. An example of a CISC<br />
design is the <strong>IBM</strong> System/360 and its successors.<br />
<strong>The</strong> need for complex and highly functional instructions primarily existed<br />
because computers were equipped with small quantities of slow memory.<br />
Complex instructions resulted in fewer instructions per program, so less memory<br />
was needed for program storage. Complex instructions were also easier for<br />
human programmers to use when developing applications in assembly language<br />
as single instructions could be rich in functionality.<br />
However, studies showed that only a small percentage of CISC instructions were<br />
commonly used by programs. Also, the benefit of the CISC approach diminished<br />
over time as memory became larger and faster, and high-level languages<br />
replaced assembly language.<br />
<strong>The</strong> RISC concept was originally developed by <strong>IBM</strong> Fellow John Cocke in the<br />
early 1970s and includes the following characteristics:<br />
► A simple architecture with an optimized set of machine instructions<br />
<strong>The</strong> instructions in a RISC architecture are typically simple in function and<br />
regular in size. This simplifies the design of the instruction decode hardware.<br />
<strong>The</strong> large number of infrequently used instructions typically found in CISC<br />
architectures are eliminated and replaced by sequences of simple<br />
instructions in RISC architectures.<br />
► A high instruction execution rate<br />
A common objective of early RISC processor designs was to execute an<br />
average of one instruction per processor clock cycle. This has subsequently<br />
been improved upon in superscalar RISC processor designs that can execute<br />
multiple instructions per processor clock cycle.<br />
30 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
► Optimizing compiler availability<br />
RISC architectures assume the availability of good optimizing compilers, and<br />
that most users will be developing applications in high-level languages. This<br />
has simplified the hardware design by moving some of the complexity that<br />
was previously in the hardware into the compiler. However, the simplified<br />
instruction set of RISC architectures often makes it easier for compilers to<br />
choose optimal instruction sequences.<br />
► Load and store architecture<br />
RISC architectures typically incorporate large numbers of processor registers,<br />
and computational instructions manipulate values only in these registers.<br />
Separate instructions are provided for loading data from memory and storing<br />
data to memory. Compilers can take advantage of this clean separation and<br />
schedule instructions in a way that keeps the processor busy despite the long<br />
latency associated with memory access.<br />
Today the difference between CISC and RISC is less clear. Many CISC<br />
architectures have adopted many RISC concepts. Meanwhile, the instruction<br />
sets of many RISC architectures have grown to include hundreds of instructions.<br />
<strong>The</strong> original POWER and POWER2 processors<br />
<strong>The</strong> first implementation of POWER architecture released by <strong>IBM</strong> was the<br />
POWER processor as used in the first <strong>IBM</strong> RS/6000 servers and workstations.<br />
This processor was a multiple-chip implementation of the POWER architecture<br />
comprised of either seven or nine separate chips. It featured an 8 KB instruction<br />
cache and either a 32 KB or 64 KB data cache. Over the life of the POWER<br />
processor, it was shipped with clock speeds ranging from 20 MHz to 62.5 MHz.<br />
<strong>The</strong> original POWER processor introduced the concept of a superscalar RISC<br />
processor that could perform multiple operations in a single clock cycle by<br />
exploiting multiple execution units that operated concurrently. At the time, this<br />
represented a significant improvement over contemporary processor<br />
implementations available from other manufacturers.<br />
<strong>The</strong> floating-point performance of the POWER processor was further enhanced<br />
by the presence of the fused multiply-add (FMA) instruction. It enabled two<br />
floating-point operations to be executed by the floating-point execution unit each<br />
processor clock cycle. <strong>The</strong> peak floating-point performance of the POWER<br />
processor was therefore twice the processor clock rate. For example, a POWER<br />
processor operating at 25 MHz had a peak floating-point performance of 50<br />
MFLOPS.<br />
In subsequent years, <strong>IBM</strong> introduced a single chip implementation of the<br />
POWER processor called the RISC Single Chip (RSC), which was used in the<br />
<strong>IBM</strong> RS/6000® Model 220 and Model 230.<br />
Chapter 2. Hardware components 31
In 1993, <strong>IBM</strong> announced the POWER2 processor, which expanded upon the<br />
POWER processor superscalar concept by adding additional execution units to<br />
increase the number of operations that could be performed concurrently in a<br />
single clock cycle. This processor was a multiple chip implementation of a<br />
super-set of the original POWER architecture.<br />
<strong>The</strong> POWER2 processor had two floating-point execution units, each able to<br />
execute a FMA instruction every clock cycle. <strong>The</strong>refore, the peak floating-point<br />
performance of the POWER2 processor was four times the processor clock rate.<br />
For example, a POWER2 processor operating at 55 MHz had a peak<br />
floating-point performance of 220 MFLOPS.<br />
<strong>The</strong> memory subsystem of the POWER2 was also enhanced in order to keep up<br />
with the increased level of processor performance. This was achieved using<br />
larger caches and wider paths to memory.<br />
As a super-set of the POWER architecture, the POWER2 supported several<br />
instructions that are not present on any other POWER or PowerPC Architecture<br />
processors.<br />
<strong>IBM</strong> subsequently delivered a single chip implementation of the POWER2 called<br />
the POWER2 Super Chip (P2SC), which was offered at processor clock speeds<br />
as high as 160 MHz, and delivered peak floating-point performance of 640<br />
MFLOPS.<br />
PowerPC Architecture<br />
In 1991, <strong>IBM</strong> teamed with Apple Computer and Motorola to define the PowerPC<br />
Architecture. <strong>The</strong> goals of this new architecture were to:<br />
► Permit a broad range of implementations, from low-cost controllers to<br />
high-performance processors<br />
► Be sufficiently simple to permit the design of processors that have a very<br />
short cycle time<br />
► Minimize the effects that hinder the design of aggressive superscalar<br />
implementations.<br />
► Include multiprocessor features.<br />
► Define a 64-bit architecture that is a super-set of the 32-bit POWER<br />
architecture, providing application binary compatibility for 32-bit applications<br />
While based on the POWER architecture, the PowerPC Architecture<br />
incorporated several modifications to enable it to be more widely applied in a<br />
variety of application scenarios. This vision has been subsequently realized with<br />
processors implementing PowerPC Architecture now having been installed in<br />
32 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
desktop, server, and embedded systems across commercial, consumer,<br />
industrial, and scientific settings.<br />
To achieve the design goals for the PowerPC Architecture, some features of the<br />
original POWER architecture were removed. <strong>The</strong>se were mostly features that<br />
were infrequently used. None of the POWER2 extensions to the original POWER<br />
architecture were included in the PowerPC Architecture.<br />
<strong>The</strong> PowerPC Architecture defines both 32-bit and 64-bit modes of operation.<br />
<strong>The</strong> primary differences in these two modes of operation are in the effective<br />
length of addresses used by the processor, and the availability of extra<br />
capabilities to manipulate double word (64-bit) fixed-point operands in 64-bit<br />
mode. Floating-point capabilities are the same in both 32-bit and 64-bit modes.<br />
<strong>The</strong> 32-bit PowerPC Architecture implementations only support the 32-bit mode<br />
of operation. <strong>The</strong> 64-bit PowerPC Architecture implementations support both the<br />
32-bit and 64-bit modes of operation. This enables the 64-bit PowerPC<br />
Architecture implementations to support the full-speed execution of existing<br />
32-bit applications, alongside 64-bit applications, in the same operating<br />
environment.<br />
<strong>The</strong> PowerPC Architecture is documented in a series of books. You can<br />
download the most current version of the PowerPC Architecture books from the<br />
<strong>IBM</strong> developerWorks® Web site at:<br />
http://www.ibm.com/developerworks/eserver/articles/archguide.html<br />
<strong>The</strong> architecture is divided into three parts, each one documented in a separate<br />
book:<br />
► Book I: PowerPC User Instruction Set Architecture<br />
► Book II: PowerPC Virtual Environment Architecture<br />
► Book III: PowerPC Operating Environment Architecture<br />
Book IV is also available for each specific PowerPC processor implementation. It<br />
describes extensions to the architecture that are specific to a particular<br />
implementation.<br />
We briefly review the PowerPC User Instruction Set Architecture, since this is the<br />
part of the architecture that is of most interest to application programmers.<br />
PowerPC User Instruction Set Architecture<br />
<strong>The</strong> logical components of a PowerPC processor from the perspective of a<br />
application program are:<br />
► Fixed-point processor (FXU)<br />
► Floating-point processor (FPU)<br />
► Branch processor (BPU)<br />
Chapter 2. Hardware components 33
<strong>The</strong>se components all interact with storage as illustrated in Figure 2-8.<br />
Instructions<br />
from Storage<br />
Fixed-Point<br />
Instructions<br />
Fixed-Point<br />
Processing<br />
(FXU)<br />
Figure 2-8 PowerPC logical processing model<br />
A PowerPC Architecture groups the instructions that an application program uses<br />
on a PowerPC processor along the same lines into:<br />
► Fixed-point instructions<br />
► Floating-point instructions<br />
► Branch instructions<br />
All instructions in PowerPC Architecture are four bytes (one word) in size and are<br />
aligned on word boundaries. This simplifies the decoding of instructions and is<br />
characteristic of a RISC architecture.<br />
<strong>The</strong> fixed-point instructions operate on a set of 32 general purpose registers<br />
(GPRs), that are each 8 bytes (64 bits) in size on 64-bit PowerPC Architecture<br />
processors. <strong>The</strong>re is also a fixed-point exception register (XER). In 32-bit mode,<br />
only the lower 32 bits of the GPRs are considered significant, and the double<br />
word load and store instructions are not available.<br />
<strong>The</strong> floating-point instructions operate on a set of 32 floating-point registers<br />
(FPRs), that are each 8 bytes (64 bits) in size. <strong>The</strong>re is also a floating-point<br />
status and control register (FPSCR). <strong>The</strong> floating point capabilities are the same<br />
in both 32-bit and 64-bit mode. <strong>The</strong> large number of GPRs and FPRs is also<br />
typical of a RISC architecture.<br />
34 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
Branch<br />
Processing<br />
(BPU)<br />
Data to/from<br />
Storage<br />
Storage<br />
Floating-Point<br />
Instructions<br />
Floating-<br />
Point<br />
Processing<br />
(FPU)
<strong>The</strong> architecture also features a condition register that is set by the result of fixed<br />
or floating point computational instructions and that can be used to control<br />
branch processing. Branch instructions also make use of a count and a link<br />
register.<br />
Figure 2-9 illustrates the complete set of architected PowerPC registers used by<br />
application programs.<br />
CR<br />
0 31<br />
LR<br />
0 63<br />
CTR<br />
0 63<br />
GPR0<br />
GPR1<br />
...<br />
GPR31<br />
0 63<br />
XER<br />
0 63<br />
FPR0<br />
FPR1<br />
...<br />
FPR31<br />
0 63<br />
FPSCR<br />
0 31<br />
Condition Register<br />
Link Register<br />
Count Register<br />
General Purpose Registers<br />
Figure 2-9 PowerPC registers used by application programs<br />
Fixed-Point Exception Register<br />
Floating-Point Registers<br />
Floating-Point Status<br />
and Control Register<br />
In the PowerPC Architecture, fixed and floating-point computational instructions<br />
can operate on operands stored in registers. Data must be loaded from memory<br />
into a register using a fixed or floating-point load instruction before computations<br />
can be performed. Similarly, data must be stored from a register into memory<br />
using an explicit fixed or floating-point store instruction once computation is<br />
completed. This behavior is also characteristic of a RISC architecture.<br />
Chapter 2. Hardware components 35
<strong>IBM</strong> processors implementing PowerPC Architecture<br />
Since <strong>IBM</strong> teamed with Motorola and Apple Computer to define PowerPC<br />
Architecture, <strong>IBM</strong> has developed many different processors that implement this<br />
architecture. <strong>The</strong>se processors fall into two main categories:<br />
► Processors that are used as the main processor within <strong>IBM</strong> workstations and<br />
servers<br />
► Processors that <strong>IBM</strong> produces for embedding in a wide variety of computing<br />
and non-computing devices<br />
We focus our review of <strong>IBM</strong> processors implementing PowerPC Architecture on<br />
those processors that are used as the main processors within <strong>IBM</strong> workstations<br />
and servers.<br />
32-bit PowerPC processors<br />
In the early 1990s, <strong>IBM</strong> jointly designed with Motorola a series of 32-bit PowerPC<br />
processors with the designation 60X that were used in <strong>IBM</strong> RS/6000<br />
workstations and servers. <strong>The</strong>y were also used in computer systems from other<br />
computer manufacturers such as Apple Computer.<br />
<strong>The</strong> first of these was the PowerPC 601®. It debuted in the <strong>IBM</strong> RS/6000 Model<br />
250 in 1993. <strong>The</strong> PowerPC 601 was an evolution of the RISC Single Chip<br />
implementation of the POWER processor and did not implement the full<br />
PowerPC Architecture. It represented a transition between the earlier POWER<br />
architecture and the PowerPC Architecture.<br />
<strong>The</strong> PowerPC 601 was subsequently followed by the PowerPC 603 and<br />
PowerPC 604. <strong>The</strong>se represented full implementations of the 32-bit PowerPC<br />
Architecture. Enhanced versions of these processors known as the PowerPC<br />
603e and PowerPC 604e were subsequently introduced.<br />
<strong>The</strong> PowerPC 603 was an entry-level processor. <strong>The</strong> PowerPC 604 was a<br />
high-end processor and introduced support for symmetrical multi-processor<br />
(SMP) configurations.<br />
RS64 processors<br />
In 1997, <strong>IBM</strong> introduced its first servers that incorporated processors based on<br />
the 64-bit PowerPC Architecture. <strong>The</strong>se processors were called the RS64. <strong>The</strong><br />
RS64 was subsequently enhanced with the introduction of the RS64-II, RS64-III,<br />
and RS64-IV in later years. <strong>The</strong> RS64-IV operated at processor clock speeds as<br />
high as 750 MHz.<br />
<strong>The</strong> RS64 processor and its successors were optimized for use in commercial<br />
applications and de-emphasized the floating point capabilities of the earlier<br />
POWER2 processors. <strong>The</strong> RS64 processors were used in both <strong>IBM</strong> RS/6000<br />
36 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
(later <strong>IBM</strong> Eserver pSeries®) and <strong>IBM</strong> AS/400® (later <strong>IBM</strong> Eserver iSeries)<br />
servers, most frequently in SMP configurations, with some as large as 24<br />
processors.<br />
POWER3 and POWER3-II processors<br />
<strong>The</strong> POWER3 processor was first used in <strong>IBM</strong> RS/6000 servers and<br />
workstations introduced in 1998. <strong>The</strong> POWER3 processor continued the<br />
evolution of the excellent floating-point performance provided by the POWER2<br />
Super Chip (P2SC) processor, while implementing 64-bit PowerPC Architecture<br />
and supporting SMP configurations of up to 16 processors.<br />
Upon introduction, the POWER3 processor operated at 200 MHz, and delivered<br />
peak floating-point performance of 800 MFLOPS per processor. An improved<br />
version of the POWER3 processor, known as the POWER3-II, was subsequently<br />
introduced. <strong>The</strong> POWER3-II featured processor clock speeds as high as 450<br />
MHz, and delivered peak floating point performance of 1.8 GFLOPS per<br />
processor.<br />
You can learn more about the POWER3 processor in the <strong>IBM</strong> white paper<br />
POWER3: Next Generation 64-bit PowerPC Processor Design, which is available<br />
on the Web at:<br />
http://www.ibm.com/servers/eserver/pseries/hardware/whitepapers/power3wp.html<br />
POWER4 and POWER4+ processors<br />
<strong>The</strong> POWER4 processor combined the best attributes of both the RS64 and<br />
POWER3 processors. It delivers a 64-bit PowerPC Architecture processor that is<br />
capable of delivering high levels of performance in both commercial and scientific<br />
applications.<br />
<strong>The</strong> POWER4 processor is the first PowerPC Architecture processor to feature<br />
multiple processor cores on a single chip. Each POWER4 processor chip has<br />
two independent processor cores and a shared level-2 cache.<br />
<strong>The</strong> original POWER4 processor operated at processor clock speeds as high as<br />
1.3 GHz and has featured in SMP configurations as large as 32 processor cores.<br />
An enhanced version of the POWER4 processor, known as the POWER4+, was<br />
subsequently introduced and installed at processor clock speeds as high as<br />
1.9 GHz in the 32 processor core <strong>IBM</strong> Eserver pSeries Model 690.<br />
You can learn more about the POWER4 processor in the <strong>IBM</strong> white paper<br />
entitled POWER4 System Microarchitecture, which is available on the Web at:<br />
http://www.ibm.com/servers/eserver/pseries/hardware/whitepapers/power4.html<br />
Chapter 2. Hardware components 37
PowerPC 970 and PowerPC 970FX processors<br />
<strong>The</strong>se processors were first used by <strong>IBM</strong> in the <strong>BladeCenter</strong> <strong>JS20</strong> that is the<br />
subject of this book. We explore these processors in more detail in 2.3,<br />
“PowerPC 970 and PowerPC 970FX Microprocessors” on page 29.<br />
POWER5 processor<br />
<strong>IBM</strong> recently introduced the POWER5 processor in the <strong>IBM</strong> Eserver i5 and p5<br />
families of servers.<br />
<strong>The</strong> POWER5 processor represents an evolution of the POWER4 processor.<br />
Like the POWER4, the POWER5 processor implements two processor cores on<br />
a single chip with a shared level-2 cache.<br />
<strong>The</strong> major innovation in the POWER5 processor is the introduction of<br />
symmetrical multi-threading (SMT) capabilities. SMT enables two threads of<br />
execution (contexts) to execute simultaneously on the same processor core. This<br />
can improve the performance of many applications through the exploitation of<br />
otherwise idle execution units within the processor core.<br />
<strong>The</strong> other significant innovation in the POWER5 processor design is the<br />
integration of a memory controller onto the processor chip to decrease the<br />
latency to memory and improve system reliability.<br />
2.3.2 Vector/SIMD Multimedia eXtension<br />
<strong>The</strong> Vector/SIMD Multimedia eXtension (VMX) is an extension to the PowerPC<br />
Architecture. It defines additional registers and instructions to support<br />
single-instruction multiple-data (SIMD) operations that accelerate data-intensive<br />
tasks.<br />
<strong>The</strong> VMX extensions to the PowerPC Architecture were developed jointly by<br />
Apple Computer, <strong>IBM</strong>, and Motorola. Apple Computer and Motorola use different<br />
terminology to refer to the VMX extensions of the PowerPC Architecture.<br />
Specifically, Motorola uses the term Altivec, and Apple uses the term Velocity<br />
Engine.<br />
A short vector processing history<br />
<strong>The</strong> basic concept behind vector processing is to enhance the performance of<br />
data-intensive applications by providing hardware support for operations that can<br />
manipulate an entire vector (or array) of data in a single operation. <strong>The</strong> number<br />
of data elements operated upon at a time is called the vector length.<br />
38 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Scalar processors perform operations that manipulate single data elements such<br />
as fixed-point or floating-point numbers. For example, scalar processors usually<br />
have an instruction that adds two integers to produce a single-integer result.<br />
Vector processors perform operations on multiple data elements arranged in<br />
groups called vectors (or arrays). For example, a vector add operation to add two<br />
vectors performs a pairwise addition of each element of one source vector with<br />
the corresponding element of the other source vector. It places the result in the<br />
corresponding element of the destination vector. Typically a single vector<br />
operation on vectors of length n is equivalent to performing n scalar operations.<br />
Figure 2-10 illustrates the difference between scalar and vector operations.<br />
Scalar Add Operation<br />
7<br />
1<br />
4 + 7 11<br />
6<br />
3<br />
3<br />
10<br />
+<br />
Figure 2-10 Scalar and vector operations<br />
Vector Add Operation<br />
Processor designers are continually looking for ways to improve application<br />
performance. <strong>The</strong> addition of vector operations to a processor architecture is one<br />
method that a processor designer can use to make it easier to improve the peak<br />
performance of a processor. However, the actual performance improvements that<br />
can be obtained for a specific application depend on how well the application can<br />
exploit vector operations.<br />
<strong>The</strong> concept of vector processing has existed since the 1950s. Early<br />
implementations of vector processing (known as array processing) were installed<br />
in the 1960s. <strong>The</strong>y used special purpose peripherals attached to general<br />
purpose computers. An example is the <strong>IBM</strong> 2938 Array Processor, which could<br />
be attached to some models of the <strong>IBM</strong> System/360. This was followed by the<br />
<strong>IBM</strong> 3838 Array Processor in later years.<br />
By the mid-1970s, vector processing became an integral part of the main<br />
processor in large supercomputers manufactured by companies such as Cray<br />
Research. By the mid-1980s, vector processing became available as an optional<br />
feature on large general-purpose computers such as the <strong>IBM</strong> 3090.<br />
In the 1990s, developers of microprocessors used in desktop computers adapted<br />
the concept of vector processing to enhance the capability of their<br />
microprocessors when running desktop multimedia applications. <strong>The</strong>se<br />
5<br />
6<br />
11<br />
4<br />
5<br />
2<br />
12<br />
7<br />
17<br />
7<br />
8<br />
12<br />
Chapter 2. Hardware components 39
capabilities were usually referred to as Single Instruction Multiple Data (SIMD)<br />
extensions and operated on short vectors. Examples of SIMD extensions in<br />
widespread use today include:<br />
► Intel Multimedia Extensions (MMX)<br />
► Intel Streaming SIMD Extensions (SSE)<br />
► AMD 3DNow!<br />
► Motorola Altivec and <strong>IBM</strong> VMX<br />
<strong>The</strong> SIMD extensions found in microprocessors used in desktop computers<br />
operate on short vectors of length 2, 4, 8, or 16. This is in contrast to the classic<br />
vector supercomputers that can often exploit long vectors of length 64 or more.<br />
VMX extensions to PowerPC Architecture<br />
<strong>The</strong> VMX extensions to PowerPC Architecture add a vector processor (VXU) to<br />
the PowerPC logical processing model that was described earlier in “PowerPC<br />
User Instruction Set Architecture” on page 33. This is illustrated in Figure 2-11.<br />
Fixed-Point<br />
Instructions<br />
Fixed-Point<br />
Processing<br />
(FXU)<br />
Instructions<br />
from Storage<br />
Figure 2-11 PowerPC with VMX logical processing model<br />
<strong>The</strong> VXU operates on vectors that are a total of 128-bits long. <strong>The</strong>se can be<br />
interpreted by the VXU as either:<br />
► A vector of sixteen 8-bit bytes<br />
► A vector of eight 16-bit half words<br />
► A vector of four 32-bit words<br />
40 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
Branch<br />
Processing<br />
(BPU)<br />
Floating-Point<br />
Instructions<br />
Floating-<br />
Point<br />
Processing<br />
(FPU)<br />
Storage<br />
Data to/from<br />
Storage<br />
Vector<br />
Processing<br />
(VXU)<br />
Vector<br />
Instructions
<strong>The</strong> VMX extensions to PowerPC Architecture define 32 vector registers that<br />
form the vector register file (VRF). <strong>The</strong> VRF is architecturally distinct from the<br />
standard PowerPC FPRs and GPRs.<br />
<strong>The</strong> VMX extensions to PowerPC also define two additional registers:<br />
► <strong>The</strong> VMX status and control register (VSCR), which is used to control the<br />
operation of the VXU and report the status of some VMX operations<br />
► <strong>The</strong> VRSAVE register, which can be used to assist the operating system save<br />
state across context switches by providing a mechanism for software to<br />
indicate what vector registers are in use<br />
<strong>The</strong> additional registers provided by VMX are illustrated in Figure 2-12.<br />
VRSAVE<br />
0 31<br />
VSCR<br />
0 31<br />
Figure 2-12 VMX registers<br />
VRSAVE Register<br />
Vector Status and Control Register<br />
Vector Register File (VRF)<br />
VR0<br />
VR1<br />
VR31<br />
0 127<br />
<strong>The</strong> VMX extensions to PowerPC Architecture define new instructions that use<br />
the VXU to manipulate vectors stored in the VRF. <strong>The</strong>se instructions fall into<br />
these categories:<br />
► Vector integer arithmetic instructions (on 8-bit, 16-bit, or 32-bit integers)<br />
► Vector floating-point arithmetic instructions (32-bit only)<br />
► Vector load and store instructions<br />
► Vector permutation and formatting instructions<br />
► Processor control instructions used to read and write from the VMX status<br />
and control register<br />
► Memory control instructions used to manage caches<br />
...<br />
Chapter 2. Hardware components 41
For additional information about the topics presented in this chapter, we<br />
recommend that you read the PowerPC Microprocessor Family: AltiVec<br />
Technology Programming Environments Manual. You can find this manual on the<br />
Web at:<br />
http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/FBFA164F824370F987256<br />
D6A006F424D<br />
2.3.3 Features of the PowerPC 970 and PowerPC 970FX<br />
<strong>The</strong> <strong>IBM</strong> PowerPC 970 Microprocessor is a recent implementation of 64-bit<br />
PowerPC Architecture designed for use in workstations and small servers<br />
requiring from one to four processors. <strong>The</strong> PowerPC 970 is based on the<br />
processor core used in the POWER4 with the addition of a VXU to support the<br />
VMX extensions to the PowerPC Architecture.<br />
<strong>The</strong> <strong>IBM</strong> PowerPC 970FX RISC Microprocessor is an enhanced version of the<br />
PowerPC 970 that takes advantage of improvements in semiconductor<br />
manufacturing processes to deliver higher performance and lower power<br />
consumption. In most respects, the PowerPC 970FX is identical to the PowerPC<br />
970.<br />
Table 2-3 lists the physical attributes of the PowerPC 970 and PowerPC 970FX.<br />
Table 2-3 Physical attributes of PowerPC 970 and PowerPC 970FX<br />
Physical attribute PowerPC 970 PowerPC 970FX<br />
Process geometry 130 nm 90 nm<br />
Physical size 125 mm 2<br />
Unless otherwise noted, the following discussion applies to both the PowerPC<br />
970 and PowerPC 970FX. References to PowerPC 970 also refer to the<br />
PowerPC 970FX.<br />
Figure 2-13 illustrates the internal structure of the PowerPC 970.<br />
42 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
66 mm 2<br />
Transistor count 58 million 58 million<br />
Package pins 576 576
Figure 2-13 PowerPC 970 internal organization<br />
Chapter 2. Hardware components 43
<strong>The</strong> major features of the PowerPC 970 include:<br />
► Speculative, superscalar RISC processor core with vector extensions<br />
► On-chip level-1 and level-2 caches<br />
► Deeply pipelined design up to 25 stages long<br />
► Aggressive branch prediction<br />
► In order dispatch of up to five operations per cycle<br />
► Out of order issue of up to ten operations per cycle<br />
► In order completion of up to five operations per cycle<br />
► Register renaming<br />
► Up to 215 instructions in-flight<br />
► Fast selective flush of incorrect speculative instructions and results<br />
► Hardware prefetching of up to eight streams including four VMX streams<br />
We elaborate on some of these features in the following sections.<br />
Cache structure<br />
<strong>The</strong> PowerPC 970 includes several on-chip caches to reduce memory latency<br />
when fetching instructions and performing data load and store operations.<br />
Understanding the structure of these caches is sometimes useful when doing<br />
low-level assembly language programming.<br />
<strong>The</strong> on-chip caches include:<br />
► 64 KB, direct-mapped instruction cache<br />
► 32 KB, 2-way set associative data cache<br />
► 128-entry, instruction effective to real address translation (ERAT) cache<br />
instructions<br />
► 128-entry, data ERAT cache<br />
► 64-entry, fully associative segment lookaside buffer (SLB)<br />
► 1024-entry, 4-way set associative translation lookaside buffer (TLB)<br />
► 512 KB, 8-way set associative, level-2 cache<br />
Speculative superscalar features<br />
<strong>The</strong> design of the PowerPC 970 uses a variety of techniques to enable<br />
superscalar operation, where multiple instructions are executed during each<br />
processor clock cycle. This capability is enabled by the exploitation of multiple<br />
execution units within the processor core.<br />
44 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
<strong>The</strong> speculative superscalar techniques used in the PowerPC 970 include:<br />
► Aggressive branch prediction<br />
– Prediction for up to two branches per cycle<br />
– Support for up to 16 predicted branches in flight<br />
– Prediction support for both branch direction and branch addresses<br />
► In order dispatch of up to five operations into a distributed issue queue<br />
structure per cycle<br />
► Out of order issue of up to ten operations into ten execution pipelines per<br />
cycle, comprised of:<br />
– Two load or store operations<br />
– Two fixed-point register-to-register operations<br />
– Two floating-point operations<br />
– One branch operation<br />
– One condition register operation<br />
– One VMX permute operation<br />
– One VMX arithmetic operation<br />
► Register renaming on most architected PowerPC registers including the<br />
GPRs, FPRs, VRFs, condition register fields, FPSCR, VSCR, the link register<br />
(LR), and the count register (CTR)<br />
<strong>The</strong> total number of in-flight instructions within the processor core at any point in<br />
time can be as high as 215.<br />
<strong>The</strong> VMX execution units operate on multiple data elements concurrently.<br />
<strong>The</strong>refore, they provide a performance equivalent to multiple scalar execution<br />
units.<br />
Bus Interface Unit<br />
<strong>The</strong> interface between the PowerPC 970 and external devices is provided by the<br />
Bus Interface Unit (BIU). <strong>The</strong> interface is comprised of two unidirectional paths<br />
(one in, one out) that are each four bytes wide.<br />
<strong>The</strong> internal processor clock operates at an integral multiple of the interface<br />
speed. On the <strong>BladeCenter</strong> <strong>JS20</strong>, the processor clock operates at double the<br />
interface speed.<br />
<strong>The</strong> total aggregate transfer rate across the interface in bytes per second is four<br />
times of the processor clock speed (1/2 x 2 x 4). This equates to 6.4 GBps for<br />
processors operating at 1.6 GHz, and 8.8 GBps for processors operating at<br />
2.2 GHz.<br />
<strong>The</strong> interface is multiplexed, with both addresses and data being transferred<br />
across the same wires. <strong>The</strong>refore the peak theoretical data throughput across<br />
Chapter 2. Hardware components 45
the bus interface is slightly lower than the total aggregate bus transfer rate<br />
derived above.<br />
For more detailed information about the PowerPC 970 and PowerPC 970FX<br />
microprocessors, refer to the <strong>IBM</strong> Microelectronics Division Web site at:<br />
http://www.chips.ibm.com<br />
To access the documentation, follow these steps:<br />
1. Point your Internet browser to:<br />
http://www.chips.ibm.com<br />
2. Click Support.<br />
3. Click Documentation.<br />
4. Under Browse a product family, click PowerPC.<br />
5. Under Categories, click PowerPC 9XX Microprocessors.<br />
6. Under Categories, click PowerPC 970 and 970FX Microprocessors.<br />
Various documents describe the PowerPC Architecture, programming, and 64-bit<br />
computing.<br />
46 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Chapter 3. Software environment<br />
3<br />
This chapter discusses the software environments that are available for the <strong>IBM</strong><br />
Eserver <strong>BladeCenter</strong> <strong>JS20</strong>. It covers the operating systems and <strong>IBM</strong><br />
middleware supported on the <strong>BladeCenter</strong> <strong>JS20</strong>, as well as the system<br />
management tools for the <strong>JS20</strong>.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 47
3.1 Operating system support<br />
3.1.1 AIX<br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> is supported by the AIX and Linux operating systems. At<br />
the time of this writing, AIX 5.2H, AIX 5.3, Red Hat Enterprise Linux 3.0, and<br />
SUSE LINUX Enterprise Server (SLES) are the supported operating systems on<br />
this platform.<br />
Both AIX and Linux operating systems work with a 64-bit kernel and mostly 32-bit<br />
userland programs. Some of the included programs were compiled for 64 bit<br />
when the code fully exploited the 64-bit address space.<br />
AIX has evolved from its beginnings on the <strong>IBM</strong> RT to becoming the operating<br />
system of choice for <strong>IBM</strong>’s largest UNIX servers. AIX is an enterprise operating<br />
system that scales from workstations all the way up to massively parallel super<br />
computers.<br />
Certified as C2 security compliant by the US government, AIX also supports<br />
industry-standard security features such as Pluggable Authentication Modules<br />
(PAM), authenication by x.509 certificates, OpenSSH, Kerberos, and LDAP. AIX<br />
fully exploits the features of the <strong>JS20</strong> with its 64-bit kernel and support for over<br />
16 TB of disk space.<br />
With AIX installed, the <strong>JS20</strong> can run thousands of software titles that were<br />
written for the AIX platform, taking full advantage of AIX’s rich capabilities. At the<br />
time of the writing of this book, AIX Version 5.2H and AIX 5.3 support the<br />
<strong>BladeCenter</strong> <strong>JS20</strong>.<br />
3.1.2 Red Hat Enterprise Linux<br />
Red Hat has made its distribution of Linux available since 1994. Originally, Red<br />
Hat offered its distribution for free download, and sold support contracts. In 2002,<br />
Red Hat began marketing Red Hat Enterprise Linux (RHEL).<br />
Unlike the original Red Hat distribution, RHEL was available only with a<br />
maintenance contract from Red Hat. RHEL runs on x86_64, i386, ia64, ppc/64,<br />
s390, and s390x platforms.<br />
By early 2003, Red Hat stopped developing its non-enterprise distribution and<br />
focused its efforts on RHEL. Red Hat elected to turn development of its free<br />
software over to the community. This free community-supported distribution<br />
became known as Fedora.<br />
48 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Red Hat Enterprise Linux 3 - Update 2 and later is supported on the <strong>JS20</strong>.<br />
Although VMX exploitation was not supported at the time of this book’s<br />
development in the version of GNU C Compiler (gcc) that ships with RHEL 3,<br />
kernel support is available for programs that have been written to exploit VMX<br />
instructions.<br />
3.1.3 SUSE LINUX Enterprise Server<br />
SuSE has been a Linux distributor since 1992 and was acquired by Novell in<br />
early 2004. Originally, SuSE provided a free downloadable distribution, for which<br />
support contracts could be purchased as desired. Eventually, their product line<br />
was split. SuSE now markets SUSE LINUX Desktop, as well as SUSE LINUX<br />
Enterprise Server.<br />
SUSE LINUX Desktop is free to download, where SLES is available only with the<br />
purchase of a maintenance contract. <strong>The</strong> desktop product is also available only<br />
for x86_64 and IA32, where SLES supports IA32, ia64, ppc/64, s390, s390x, and<br />
x86_64.<br />
SLES Version 8 and Version 9 are supported on the <strong>BladeCenter</strong> <strong>JS20</strong>. SLES 9,<br />
the latest version at the time of writing, ships with the 2.6 Linux kernel. This<br />
brings significant gains to performance and scalability.<br />
Both versions of SLES are VMX savvy and have versions of gcc that allow<br />
manual exploitation of the vector engine.<br />
3.2 System management tools<br />
<strong>IBM</strong> has devised a strategy for systems management called the Universal<br />
Manageability Initiative. <strong>The</strong>re are several technologies that have been<br />
incorporated under this umbrella. <strong>The</strong>y all work toward the same goal: simple,<br />
effective solutions for managing heterogeneous environments.<br />
One of the main building blocks in this strategy is the use of simple yet powerful<br />
software to oversee the management of many different systems. This section<br />
investigates some of the possible options that are available for managing such<br />
systems. It reviews the <strong>BladeCenter</strong> Web interfaces, <strong>IBM</strong> Director, Cluster<br />
Systems Management, and xCAT.<br />
3.2.1 <strong>BladeCenter</strong> Web interfaces<br />
<strong>The</strong> <strong>BladeCenter</strong> Web interface allows system administrators to easily and<br />
effectively manage up to 14 blades from an integrated interface. From trivial tasks<br />
such and powering blades on or off, to more complex tasks such as firmware<br />
Chapter 3. Software environment 49
3.2.2 <strong>IBM</strong> Director<br />
management, the Web interface allows powerful control over all blades and<br />
input/output (I/O) modules that are attached to the <strong>BladeCenter</strong> chassis.<br />
Management of other <strong>BladeCenter</strong> resources, such as I/O modules, is also<br />
possible from here, as well as the retrieval of system health information.<br />
<strong>BladeCenter</strong>-specific features are also configured from here such as the Serial<br />
over LAN (SoL).<br />
<strong>IBM</strong> Director assists with the remote management of many <strong>IBM</strong> and non-<strong>IBM</strong><br />
machines including the <strong>BladeCenter</strong> <strong>JS20</strong>. <strong>The</strong> <strong>IBM</strong> Director console allows<br />
system administrators to manage multiple <strong>BladeCenter</strong> chassis from a common<br />
interface. It is the ideal solution for administering <strong>BladeCenter</strong> chassis in<br />
heterogeneous environments or environments where a Director infrastructure<br />
exists.<br />
From the <strong>IBM</strong> Director console, system administrators can monitor resource<br />
utilization. This key feature can be used for performance and capacity planning,<br />
as well as to alert support staff of critical errors such as hardware failure.<br />
<strong>IBM</strong> Director also allows the remote management of software. You can remotely<br />
reconcile and install software from the console interface. This can be useful in<br />
large environments for such resource-intensive tasks as patch management. By<br />
using the Software Distribution Premium Edition, you can remotely install<br />
patches to several servers at the same time with the click of a button.<br />
You can learn more about <strong>IBM</strong> Director in 8.1, “<strong>IBM</strong> Director” on page 140.<br />
3.2.3 <strong>IBM</strong> Cluster Systems Management<br />
<strong>IBM</strong> Cluster Systems Management (CSM) provides many useful functions to<br />
manage a cluster from a single point of control. <strong>The</strong>se include resource<br />
monitoring, automated monitoring and operation, remote hardware control,<br />
remote command execution, security, configuration file management, parallel<br />
network installation, and diagnostics.<br />
By consolidating these capabilities, CSM helps to increase efficiency of an<br />
administrator’s time and reduce their expenses. It helps administrators install<br />
their clusters rapidly by automating many configuration tasks and by leveraging<br />
existing open source products.<br />
CSM also provides efficient monitoring of cluster resources without<br />
overwhelming network bandwidth. <strong>The</strong> automated error detection CSM provides<br />
helps catch problems before they impact the environment, and assists with rapid<br />
50 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
esolution and recovery of problems that occur. CSM’s architecture and modular<br />
construction maximizes flexibility so cluster solutions can evolve and grow as<br />
needs change.<br />
CSM is a collection of components that have been integrated to provide the basis<br />
to construct and manage a cluster. Each of these components provides specific<br />
capabilities related to the management of the cluster. This component-based<br />
architecture provides flexibility for future expansion of the capabilities provided<br />
by CSM. Each of the CSM components can be easily personalized to help meet<br />
specific needs. For example, a cluster administrator can set up monitoring of<br />
application processes and take actions if those processes disappear.<br />
In short, CSM has been designed for large-scale cluster environments that<br />
require simple control over multiple homonymous servers.<br />
This book covers CSM in more detail in 8.2, “<strong>IBM</strong> Cluster Systems Management”<br />
on page 147.<br />
Chapter 3. Software environment 51
52 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Chapter 4. Planning considerations<br />
4<br />
This chapter discusses planning considerations associated with the<br />
implementation of the <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>. Specifically it covers<br />
network planning, operating system installation, and systems management.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 53
4.1 Network planning<br />
Successful installation of the <strong>BladeCenter</strong> <strong>JS20</strong> requires that you have a clear<br />
plan for how you will use the various networking capabilities of the <strong>BladeCenter</strong><br />
infrastructure. This plan should address the following questions:<br />
► What network connectivity is needed for the blade servers to support the<br />
applications installed on them?<br />
► What network connectivity is needed to manage the <strong>BladeCenter</strong>,<br />
input/output (I/O) modules, and blade servers?<br />
► What Virtual local area networks (VLANs) are required for each LAN Switch<br />
I/O Module to provide the network connectivity established previously?<br />
► What IP subnet will be used for each VLAN and how will IP addresses be<br />
allocated to each device on the VLAN?<br />
► Will IP addresses be assigned statically or dynamically using DHCP?<br />
► What host names will be used for each network interface?<br />
► How will host names be resolved to IP addresses?<br />
► Are there requirements for a high-performance, low-latency interconnection<br />
network?<br />
► Where multiple <strong>BladeCenter</strong> chassis are installed, how will they be<br />
interconnected?<br />
<strong>The</strong> following sections explore common network planning requirements that we<br />
expect to arise in the installation of <strong>BladeCenter</strong> <strong>JS20</strong>s.<br />
4.1.1 Minimal network requirements<br />
At a minimum, most <strong>BladeCenter</strong> <strong>JS20</strong> environments have the following network<br />
requirements:<br />
► A dedicated hardware management subnet: It is used to access both the<br />
Management Module and management interfaces of I/O modules that are<br />
installed in each <strong>BladeCenter</strong> chassis.<br />
► A Serial over LAN (SoL) subnet internal to each <strong>BladeCenter</strong> chassis that<br />
supports the SoL remote text console function: This is always implemented<br />
using a LAN Switch I/O Module installed in I/O module bay 1.<br />
► A subnet connected to each <strong>BladeCenter</strong> <strong>JS20</strong>: It is used to install and<br />
manage the operating system on the blade server. Where you run AIX on<br />
some blade servers, and Linux on other blade servers, you may need to use<br />
multiple subnets for this purpose.<br />
54 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
► One or more subnets connected to each <strong>BladeCenter</strong> <strong>JS20</strong>: <strong>The</strong>y are used by<br />
applications installed on the blade server to communicate other systems.<br />
Figure 4-1 illustrates how these requirements can be provided in a logical<br />
network view.<br />
I/O<br />
Modules<br />
I/O<br />
Module I/O<br />
Module I/O<br />
Module<br />
<strong>JS20</strong><br />
Blade<br />
Server<br />
Application<br />
Subnet<br />
Figure 4-1 Network logical view<br />
Hardware<br />
Management<br />
Subnet<br />
<strong>JS20</strong><br />
Blade<br />
Server<br />
Management<br />
Module<br />
SOL<br />
Subnet<br />
Console Console Console Console Console Console<br />
<strong>JS20</strong><br />
Blade<br />
Server<br />
<strong>JS20</strong><br />
Blade<br />
Server<br />
<strong>JS20</strong><br />
Blade<br />
Server<br />
OS<br />
Management<br />
Subnet<br />
<strong>JS20</strong><br />
Blade<br />
Server<br />
eth0 eth1 eth0 eth1 eth0 eth1 eth0 eth1 eth0 eth1 eth0 eth1<br />
<strong>The</strong> sections that follow describe each of the logical networks illustrated in<br />
Figure 4-1.<br />
Chapter 4. Planning considerations 55
Hardware management subnet<br />
We recommend that you establish a dedicated hardware management subnet.<br />
<strong>The</strong> 10/100BaseT Ethernet interface on the Management Modules installed in<br />
each <strong>BladeCenter</strong> chassis provides the gateway to this subnet for each<br />
<strong>BladeCenter</strong> chassis. You need to install external LAN switches or hubs to<br />
interconnect the 10/100BaseT Ethernet interface on the Management Modules<br />
of each <strong>BladeCenter</strong> chassis with external management systems.<br />
You use this subnet to access the Management Module Web interface and<br />
command line interface (CLI). You also use this subnet to access the Web<br />
interface and CLI of I/O modules.<br />
System management applications such as <strong>IBM</strong> Director or <strong>IBM</strong> Cluster Systems<br />
Management (CSM) also use this subnet to communicate with the hardware<br />
management functions of the <strong>BladeCenter</strong> infrastructure.<br />
Restrict access to this subnet to those management systems, system<br />
administrators, and operators who have responsibility for managing the<br />
<strong>BladeCenter</strong> infrastructure.<br />
You need to allocate multiple IP addresses to each <strong>BladeCenter</strong> chassis on this<br />
subnet, including:<br />
► One IP address for the external interface of the Management Module in each<br />
<strong>BladeCenter</strong> chassis<br />
► One IP address for the internal interface of the Management Module in each<br />
<strong>BladeCenter</strong> chassis<br />
► One IP address for the management interface of each I/O module in each<br />
<strong>BladeCenter</strong> chassis<br />
Note: Although the logical network view illustrated in Figure 4-1 shows the I/O<br />
Management Module interfaces connecting directly to the hardware<br />
management subnet, they are physically connected via the Management<br />
Module, which acts as gateway to those interfaces. <strong>The</strong> Management Module<br />
performs a proxy Address Resolution Protocol (ARP) function to make it<br />
appear as though the I/O module management interfaces are attached to the<br />
hardware management subnet.<br />
It is possible to use a different subnet for the Management Module internal<br />
network interface and I/O module management interfaces. However, we do not<br />
recommend this configuration.<br />
Learn how to configure these IP addresses in Chapter 5, “Hardware setup” on<br />
page 69.<br />
56 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Serial over LAN subnet<br />
<strong>The</strong> SoL remote text console function requires a subnet and underlying VLAN<br />
that is implemented by a LAN Switch I/O Module installed in I/O module bay 1 of<br />
the <strong>BladeCenter</strong> chassis (see Figure 2-2 on page 9). This subnet and VLAN is<br />
entirely internal to each <strong>BladeCenter</strong> chassis and should not be externally<br />
accessible.<br />
If you use the 4-Port Gigabit Ethernet Switch Module or the Nortel Networks<br />
Layer 2-7 Gigabit Ethernet Switch Module, the VLAN uses VLAN ID 4095. If you<br />
use the Cisco Systems Intelligent Gigabit Ethernet Switch Module, you can<br />
choose the VLAN ID.<br />
You need to assign a unique range of IP addresses to this subnet for use by the<br />
SoL remote text console function.<br />
Important: One IP address is required for each blade server.<br />
You need to specify only the starting IP address within the range of IP addresses<br />
that you assign into the Management Module. <strong>The</strong> Management Module then<br />
automatically assigns consecutive IP addresses from the starting address that<br />
you provide to each blade server that you have installed.<br />
You can learn how to configure the SoL subnet and VLAN in 5.5.1, “Configuring<br />
Serial over LAN” on page 89.<br />
Operating system management subnet<br />
We expect most environments that use the <strong>BladeCenter</strong> <strong>JS20</strong> to rely on the<br />
network installation procedure to install the operating systems. We discuss this<br />
further in 4.2, “Operating system installation” on page 60.<br />
<strong>The</strong> operating system management subnet is used to support both the initial<br />
installation and subsequent management of the operating systems installed on<br />
<strong>BladeCenter</strong> <strong>JS20</strong>s. This subnet is implemented using a VLAN provided by the<br />
LAN Switch I/O Modules installed in I/O module bay 2 of each <strong>BladeCenter</strong><br />
chassis. Alternatively you can implement it using a Pass-Thru I/O Module<br />
installed in I/O module bay 2 that is connected to an external LAN switch.<br />
You may want to install both AIX and Linux operating systems on different<br />
<strong>BladeCenter</strong> <strong>JS20</strong>s in the same environment. In this case, you may need to set<br />
up multiple operating system management subnets and underlying VLANs, one<br />
for blade servers running AIX and the other for blade servers running Linux.<br />
Chapter 4. Planning considerations 57
Application subnet<br />
<strong>The</strong> primary reason you install <strong>BladeCenter</strong> <strong>JS20</strong>s is to support applications.<br />
Many applications have requirements to communicate with other systems. You<br />
use one or more application subnets for this purpose.<br />
<strong>The</strong> primary application subnet is implemented using a VLAN provided by the<br />
LAN Switch I/O Modules installed in I/O module bay 1 of each <strong>BladeCenter</strong><br />
chassis. <strong>The</strong> same LAN Switch I/O Module is used to support the SoL subnet<br />
and VLAN.<br />
Where different <strong>BladeCenter</strong> <strong>JS20</strong>s participate in different applications and there<br />
is a requirement to separate application traffic, you may need to define multiple<br />
application subnets and VLANs. Each <strong>BladeCenter</strong> <strong>JS20</strong> is connected to the<br />
appropriate application subnet based on the applications that are installed on the<br />
blade server.<br />
If application communication requirements with other systems are complex, you<br />
can install an additional pair of Gigabit Ethernet interfaces on each <strong>BladeCenter</strong><br />
<strong>JS20</strong>. You do this using the Gigabit Ethernet Expansion Card in conjunction with<br />
compatible I/O modules installed in I/O module bays 3 and 4.<br />
4.1.2 High-performance, low-latency network requirements<br />
Distributed memory parallel applications may require the installation of a<br />
high-performance, low-latency interconnection network between <strong>BladeCenter</strong><br />
<strong>JS20</strong>s. This requirement can be supported through the use of an optional<br />
Myrinet network.<br />
To install a Myrinet network, you must install a Myrinet Expansion Card on each<br />
<strong>BladeCenter</strong> <strong>JS20</strong> that requires connectivity to the high-performance,<br />
low-latency interconnection network. You must also install an Optical Pass-Thru<br />
I/O Module in I/O module bay 4 of each <strong>BladeCenter</strong> chassis that contains blade<br />
servers equipped with Myrinet Expansion Cards.<br />
<strong>The</strong>n connect the Optical Pass-Thru I/O Module to external Myrinet switches to<br />
complete the Myrinet network infrastructure. This is illustrated in Figure 4-2.<br />
58 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Blade<br />
Server<br />
Myrinet<br />
Expansion<br />
Card<br />
Blade<br />
Server<br />
Myrinet<br />
Expansion<br />
Card<br />
Break-out<br />
Cables<br />
Blade<br />
Server<br />
Myrinet<br />
Expansion<br />
Card<br />
...<br />
Optical Pass-Thru Module<br />
I/O Module Bay 4<br />
External<br />
Myrinet<br />
Switch<br />
Figure 4-2 Myrinet network infrastructure<br />
<strong>The</strong> Myrinet network infrastructure can be used to support application<br />
programming interfaces such as the message passing interface (MPI) that are<br />
often used by distributed memory parallel applications.<br />
You can define IP addresses for each Myrinet network interface. You can also<br />
use the Myrinet network infrastructure to support any application communication<br />
based on IP protocols. For example, you can use this capability to support a<br />
clustered file system such as <strong>IBM</strong> General Parallel File System (GPFS).<br />
You can define a dedicated IP subnet for use with the Myrinet network<br />
infrastructure that is distinct from the other IP subnets that are identified in 4.1.1,<br />
“Minimal network requirements” on page 54.<br />
4.1.3 Multiple <strong>BladeCenter</strong> environments<br />
Blade<br />
Server<br />
Myrinet<br />
Expansion<br />
Card<br />
Blade<br />
Server<br />
Myrinet<br />
Expansion<br />
Card<br />
Blade<br />
Server<br />
Myrinet<br />
Expansion<br />
Card<br />
Large configurations of <strong>BladeCenter</strong> <strong>JS20</strong>s require the installation of multiple<br />
<strong>BladeCenter</strong> chassis. To support the network requirements identified previously,<br />
you need to install external LAN switches to interconnect the <strong>BladeCenter</strong><br />
chassis with each other.<br />
Chapter 4. Planning considerations 59
4.2 Operating system installation<br />
<strong>The</strong>re are two methods to install the <strong>JS20</strong> in a <strong>BladeCenter</strong>. <strong>The</strong> first is an<br />
attended method, using CD media. <strong>The</strong> second is a remote method through a<br />
network installation. Using the CD installation media is similar to other pSeries<br />
systems. You assign the CD-ROM to a blade, follow the installation steps<br />
answering any prompts, and then repeat the process on the next blade.<br />
Refer to the following Web site for instructions to install SuSE from CDs:<br />
http://www.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm&lndocid=<br />
MIGR-54819<br />
With network installation, you can perform several installations at the same time.<br />
<strong>The</strong> method is designed to reduce the installation time required when a large<br />
number of blades requires operating system installation. This method was<br />
chosen as the focus for the following sections.<br />
4.2.1 Network installation planning<br />
You can use two approaches to set up a network installation environment for the<br />
<strong>BladeCenter</strong> <strong>JS20</strong>:<br />
► Establish one or more network installation servers and manually initiate<br />
network installation tasks.<br />
► Use a systems management tool, such as <strong>IBM</strong> Cluster Systems Management<br />
or xCAT, which can be used to automate much of the setup of network<br />
installation servers and initiation of network installation tasks.<br />
This book provides examples of both approaches. <strong>The</strong> remainder of this section<br />
focuses on the planning considerations for the first approach. You can find<br />
planning considerations for the second approach in 4.3, “Systems management”<br />
on page 63.<br />
Required network services<br />
Network installation depends on several different network services:<br />
► A bootstrap protocol (BOOTP) server or dynamic host configuration protocol<br />
(DHCP) server<br />
► A Trivial File Transfer Protocol (TFTP) server<br />
► A Network File System (NFS) server<br />
You can use one physical server to provide all three of these network services. If<br />
you want to install AIX on <strong>BladeCenter</strong> <strong>JS20</strong>s, this server must run AIX. If you<br />
want to install Linux on the <strong>BladeCenter</strong> <strong>JS20</strong>s, we recommend that you use the<br />
60 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
same Linux distribution in the network installation server that you are planning to<br />
install on the <strong>BladeCenter</strong> <strong>JS20</strong>s.<br />
In some situations, you may want to install AIX on some <strong>BladeCenter</strong> <strong>JS20</strong>s and<br />
Linux on other <strong>BladeCenter</strong> <strong>JS20</strong>s. While it is possible to use a single AIX server<br />
to do this, we recommend that you establish two network install servers, one for<br />
installing AIX and the other for installing your chosen Linux distribution.<br />
Important: A potential problem may arise when you have multiple servers<br />
acting as BOOTP or DHCP servers. Be careful when planning your network so<br />
that they do not interfere with one another.<br />
One approach is to place the <strong>BladeCenter</strong> <strong>JS20</strong>s running AIX and the AIX<br />
network install server on a different VLAN from the <strong>BladeCenter</strong> <strong>JS20</strong>s<br />
running Linux and the Linux install server. If you use this approach, you must<br />
also disable the relay of BOOTP or DHCP requests in your network routers.<br />
To learn how to set up AIX network install servers, see Chapter 7, “Installing AIX<br />
on the <strong>JS20</strong>” on page 123. Or to set up Linux network install servers, see<br />
Chapter 6, “Installing Linux” on page 101.<br />
Setting up network installation<br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> firmware has the capability to boot an operating system<br />
over a network using the BOOTP. You use this capability to initiate network<br />
installation of the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
<strong>The</strong> BOOTP protocol is a client-server protocol. When initiating a network<br />
installation on a <strong>BladeCenter</strong> <strong>JS20</strong>, it behaves as a BOOTP client. <strong>The</strong>refore,<br />
you need to set up a BOOTP server to support initiating a network installation.<br />
When you instruct the <strong>BladeCenter</strong> <strong>JS20</strong> firmware to boot over a network using<br />
BOOTP, it sends a request to the BOOTP server. <strong>The</strong> BOOTP server should<br />
generate a response that contains the following information:<br />
► <strong>The</strong> IP address, subnet mask, and default gateway that the <strong>BladeCenter</strong> <strong>JS20</strong><br />
should use for the network interface that sent the BOOTP request<br />
► <strong>The</strong> IP address of a TFTP server that the <strong>BladeCenter</strong> <strong>JS20</strong> should contact,<br />
and the name of the file on that server that the <strong>BladeCenter</strong> <strong>JS20</strong> should<br />
request, to get the operating system installation boot image<br />
<strong>The</strong> BOOTP protocol has been around since the mid1980s. In many<br />
environments, BOOTP has now been replaced by the newer DHCP protocol.<br />
DHCP was designed to interoperate with BOOTP, and most DHCP servers can<br />
serve BOOTP clients.<br />
Chapter 4. Planning considerations 61
When you perform network installations of AIX, you use the AIX BOOTP server.<br />
When you perform network installations of Linux, you use a DHCP server that is<br />
configured to support BOOTP clients.<br />
When the <strong>BladeCenter</strong> <strong>JS20</strong> firmware has received a response from a BOOTP<br />
server, it contacts the TFTP server specified in the BOOTP response to load the<br />
operating system’s installation boot image.<br />
<strong>The</strong> operating system’s installation boot image then starts running on the<br />
<strong>BladeCenter</strong> <strong>JS20</strong>. Eventually it contacts the Network File System (NFS) server<br />
to obtain the files that it needs to perform the operating system installation.<br />
Learn the details about the actual process of setting up network installation in<br />
5.5.4, “Specifying IP parameters to Open Firmware” on page 96, and 7.1,<br />
“Minimal NIM installation” on page 124.<br />
Network interface selection<br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> has two standard Gigabit Ethernet interfaces. <strong>The</strong> first is<br />
connected to I/O module bay 1, and the second is connected to I/O module bay<br />
2. Refer to Figure 2-2 on page 9.<br />
Restriction: <strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> does not support the keyboard, video and<br />
mouse (KVM) console supported by other blade servers. Instead the<br />
<strong>BladeCenter</strong> <strong>JS20</strong> only supports SoL remote text consoles via the<br />
<strong>BladeCenter</strong> Management Module.<br />
<strong>The</strong> SoL remote text console function uses the first Gigabit Ethernet interface on<br />
each <strong>BladeCenter</strong> <strong>JS20</strong> for communication between the Management Module<br />
and the service processor found in each <strong>BladeCenter</strong> <strong>JS20</strong>. This interface is<br />
known as eth0 under Linux and en0 under AIX.<br />
Under normal conditions, using the Gigabit Ethernet interface by the SoL remote<br />
text console function is entirely transparent. It does not impact usage of the<br />
Ethernet interface by the operating system running on the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
Unfortunately, the initiation of network installation using the BOOTP protocol from<br />
the <strong>BladeCenter</strong> <strong>JS20</strong> firmware temporarily disrupts the operation of the SoL<br />
remote text console when it occurs on the same physical Gigabit Ethernet<br />
interface. This disruption makes it difficult to diagnose problems with the network<br />
installation process. <strong>The</strong>refore, we recommend that you use the second Gigabit<br />
Ethernet interface on each <strong>BladeCenter</strong> <strong>JS20</strong> during network installation of the<br />
operating system. <strong>The</strong> second interface is known as eth1 under Linux and en1<br />
under AIX.<br />
62 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
To use the second interface, you must have a LAN Switch I/O Module, or<br />
Pass-Thru I/O Module connected to an external LAN switch, installed in I/O<br />
module bay 2. You can learn more about why you should use the second Gigabit<br />
Ethernet interface for network installation in the RETAIN® tip H181655 on the<br />
Web at:<br />
http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-55282<br />
4.3 Systems management<br />
4.3.1 <strong>IBM</strong> Director<br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> supports a rich set of systems management capabilities.<br />
This section discusses planning considerations associated with exploiting some<br />
of those systems management capabilities.<br />
<strong>IBM</strong> Director is a systems management tool. It was originally developed to<br />
provide a comprehensive management solution for servers and workstations<br />
based on Intel microprocessors. Recently its capabilities have been expanded to<br />
support management of other platforms such as the <strong>BladeCenter</strong> <strong>JS20</strong>. We<br />
introduce <strong>IBM</strong> Director in 3.2.2, “<strong>IBM</strong> Director” on page 50.<br />
This section reviews the major planning considerations associated with the<br />
installation of <strong>IBM</strong> Director. For a comprehensive treatment of planning for <strong>IBM</strong><br />
Director, refer to the <strong>IBM</strong> Redbook Implementing System Management Solutions<br />
using <strong>IBM</strong> Director, SG24-6188.<br />
<strong>IBM</strong> Director has three components:<br />
► <strong>IBM</strong> Director Server: This is the core of the <strong>IBM</strong> Director system<br />
management solution.<br />
► <strong>IBM</strong> Director Console: This provides the user interface that system<br />
administrators and operators use to interact with <strong>IBM</strong> Director.<br />
► <strong>IBM</strong> Director Agents: <strong>The</strong>se provide the mechanism by which the <strong>IBM</strong><br />
Director can monitor and control a specific server and operating system<br />
environment.<br />
You can only install the <strong>IBM</strong> Director Agent on the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
Chapter 4. Planning considerations 63
<strong>IBM</strong> Director Agent for the <strong>BladeCenter</strong> <strong>JS20</strong><br />
Support for the <strong>BladeCenter</strong> <strong>JS20</strong> was introduced in <strong>IBM</strong> Director Version 4.12.<br />
You can learn how to install the <strong>IBM</strong> Director Agent in 8.1.2, “Installing an <strong>IBM</strong><br />
Director Agent” on page 145.<br />
We recommend that you install the <strong>IBM</strong> Director Server on an xSeries server.<br />
For maximum functionality, this server needs IP connectivity to the Management<br />
Module in each <strong>BladeCenter</strong> chassis and to every blade server. This connectivity<br />
can be provided by connecting the <strong>IBM</strong> Director Server to both the hardware<br />
management subnet and operating system management subnets described in<br />
4.1.1, “Minimal network requirements” on page 54.<br />
If your environment is primarily comprised of servers running Linux, consider<br />
installation of the <strong>IBM</strong> Director Server under Linux.<br />
4.3.2 <strong>IBM</strong> Cluster Systems Management<br />
<strong>IBM</strong> Cluster Systems Management is a systems management tool. It simplifies<br />
the management of clusters of servers running the AIX and Linux operating<br />
systems.<br />
This section focuses on the planning considerations that are associated with<br />
using CSM to manage <strong>BladeCenter</strong> <strong>JS20</strong>s. For more planning information, refer<br />
to the <strong>IBM</strong> Cluster Systems Management for Linux Planning and Installation<br />
Guide, SA22-7873, and the <strong>IBM</strong> Cluster Systems Management for AIX5L<br />
Planning and Installation Guide, SA22-7919.<br />
Supported CSM and operating system releases<br />
Preliminary support for the <strong>BladeCenter</strong> <strong>JS20</strong> was introduced in CSM for Linux<br />
Version 1.3.3.1.<br />
CSM requires specific support for each operating system version that is running<br />
on servers that it manages. Currently (at the time of publication) CSM Version 1.4<br />
supports the following operating systems on the <strong>BladeCenter</strong> <strong>JS20</strong>:<br />
► Red Hat Enterprise Linux Version 3 (QU3)<br />
► AIX 5L Version 5.2 and 5.3<br />
► SUSE LINUX Enterprise Server (SLES) Version 8.1and 9<br />
To check for updates to this list of supported operating systems, refer to the CSM<br />
publication site at:<br />
http://publib.boulder.ibm.com/clresctr/windows/public/clusterbooks.html<br />
64 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Management node selection<br />
Each CSM cluster requires a management node, which provides the central point<br />
of administration and control for the entire cluster. CSM has complex rules<br />
concerning the compatibility of management nodes, non-management nodes<br />
(cluster nodes), and the operating system they run. We attempt to simplify these<br />
rules by focusing exclusively on the requirements for a management node that<br />
only manages <strong>BladeCenter</strong> <strong>JS20</strong>s. If your environment is more complex, refer to<br />
the CSM planning documents.<br />
You can use a CSM management node to both install and manage <strong>BladeCenter</strong><br />
<strong>JS20</strong>s, or you can use a management node to manage <strong>BladeCenter</strong> <strong>JS20</strong>s that<br />
were installed using other mechanisms. Most environments use CSM to both<br />
install and manage <strong>BladeCenter</strong> <strong>JS20</strong>s, so we focus on that scenario.<br />
If you use a management node to both install and manage <strong>BladeCenter</strong> <strong>JS20</strong>s<br />
running Linux, you must use the same type of Linux distribution on both the<br />
management node and the <strong>BladeCenter</strong> <strong>JS20</strong>s. However, the management node<br />
does not need the same processor architecture that is used in the <strong>BladeCenter</strong><br />
<strong>JS20</strong>.<br />
For example, if you plan to install SLES on <strong>BladeCenter</strong> <strong>JS20</strong>s, you can use the<br />
following types of management node and operating system combinations to both<br />
install and manage the <strong>BladeCenter</strong> <strong>JS20</strong>s:<br />
► A supported xSeries server running SLES<br />
► A supported pSeries server running SLES<br />
► A <strong>IBM</strong> HS20 Blade Server running SLES<br />
► A <strong>IBM</strong> <strong>BladeCenter</strong> <strong>JS20</strong> running SLES<br />
In large installations, we recommend that you use either an xSeries or pSeries<br />
server as a management node. If your environment also contains<br />
non-<strong>BladeCenter</strong> <strong>JS20</strong>s, such as the HS20 Blade Server, or xSeries cluster<br />
nodes, we recommend that you use an xSeries management node such as the<br />
xSeries 345.<br />
We performed our testing of CSM using an xSeries management node, since<br />
many users of the <strong>BladeCenter</strong> <strong>JS20</strong> may have environments that includes HS20<br />
Blade Servers.<br />
Chapter 4. Planning considerations 65
Network considerations and summary<br />
<strong>IBM</strong> CSM is designed to work in the type of network environment that is outlined<br />
in 4.1, “Network planning” on page 54, with multiple subnets that segregate<br />
different types of network traffic. <strong>The</strong> CSM management node is normally<br />
equipped with multiple Ethernet network interfaces that are connected as follows:<br />
► One interface connects to the hardware management subnet to support<br />
CSM’s hardware management functions.<br />
► One interface connects to the operating system management subnet to<br />
support CSM’s capabilities to install operating systems, maintain software<br />
and configuration files, and provide ongoing monitoring of cluster nodes.<br />
► One interface connects to an external network to enable remote access to the<br />
management node by system administrators and operators.<br />
For further information about the <strong>IBM</strong> Cluster Systems Management facility, refer<br />
to the online documentation that is available.<br />
66 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Part 2 Implementing<br />
Part 2<br />
the<br />
<strong>BladeCenter</strong> <strong>JS20</strong><br />
This part discusses implementation of the <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>. It<br />
covers planning considerations and setting up hardware. It explains how to install<br />
AIX 5L and Linux. Plus it presents systems management scenarios.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 67
68 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Chapter 5. Hardware setup<br />
5<br />
This chapter explains the various activities that you must perform before you can<br />
install an operating system on the <strong>BladeCenter</strong> <strong>JS20</strong>. Such activities include<br />
configuring the Management Module and LAN switch modules and updating the<br />
firmware. <strong>The</strong>y also include configuring blade servers and the Serial over LAN<br />
(SoL) remote text console function and using the Open Firmware interfaces.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 69
5.1 Management Module configuration<br />
This section describes the steps to set up the <strong>BladeCenter</strong> Management Module<br />
so that you can work with <strong>BladeCenter</strong> <strong>JS20</strong>s. Advanced configuration of the<br />
Management Module is beyond the scope of this book.<br />
<strong>The</strong> primary setup task for the Management Module is to assign IP addresses,<br />
which are necessary to communicate with the Management Module Web and<br />
command line interfaces. To learn more about selecting these IP addresses, see<br />
Chapter 4, “Planning considerations” on page 53.<br />
<strong>The</strong> Management Module has two network interfaces, an external interface<br />
(eth0) and an internal interface (eth1). <strong>The</strong> external interface is accessible via the<br />
10/100BaseT connector on the Management Module. <strong>The</strong> internal interface is<br />
connected to the management interfaces of all the installed I/O modules that<br />
support such interfaces. This includes all switch I/O modules.<br />
<strong>The</strong> default behavior of a new Management Module is to request an IP address<br />
for the external interface via DHCP. If the Management Module does not receive<br />
a valid response from a DHCP server within two minutes of powering on, it uses<br />
a default static IP address of 192.168.70.125 with the default subnet mask of<br />
255.255.255.0.<br />
Note: <strong>The</strong> default host name is MMxxxxxxxxxxxx, where xxxxxxxxxxxx is the<br />
burned-in medium access control (MAC) address.<br />
<strong>The</strong> default IP address for the internal interface is statically assigned to be<br />
192.168.70.126 with a subnet mask of 255.255.255.0.<br />
You can reset the IP addresses of a Management Module that was previously<br />
configured back to the factory defaults by using the IP reset button shown in<br />
Figure 5-1 on the Management Module. You can find the procedure for doing this<br />
in the <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Management Module User’s Guide.<br />
70 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Figure 5-1 Management Module controls and indicators<br />
Configuring the Management Module<br />
Use the following procedure to set the IP addresses for the Management Module<br />
external and internal interfaces:<br />
1. Connect the Management Module to an isolated private Ethernet network.<br />
Also connect a workstation that has a Web browser to the same isolated<br />
private Ethernet network.<br />
2. Configure a static IP address for the workstation Ethernet interface that is in<br />
the same subnet as the Management Module default IP addresses.<br />
For example, we used a static address of 192.168.70.100 with a subnet mask<br />
of 255.255.255.0 for the workstation Ethernet interface. Do not use addresses<br />
in the range of 192.168.70.125 through 192.168.70.130. <strong>The</strong>se addresses<br />
conflict with the default addresses assigned by the Management Module.<br />
3. Connect to the Management Module Web interface by pointing your Web<br />
browser on the workstation to:<br />
http://192.168.70.125<br />
4. Enter a valid user ID and password. <strong>The</strong> factory default configuration of a<br />
Management Module defines a user ID named USERID with a password of<br />
Chapter 5. Hardware setup 71
PASSW0RD. <strong>The</strong> 0 in the password is a zero. In production environments,<br />
consider changing these defaults.<br />
5. From the Management Module Web interface, select MM Control → Network<br />
Interfaces.<br />
6. <strong>The</strong> form window shown in Figure 5-2 is displayed. Enter the desired external<br />
and internal IP addresses, subnet masks, and default gateway for the<br />
Management Module. <strong>The</strong>n click Save to store the new IP addresses.<br />
Figure 5-2 IP address configuration<br />
72 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
Note: <strong>The</strong> default user ID is USERID, and the default password is<br />
PASSW0RD. <strong>The</strong> number zero is between the W and the R.
7. Restart the Management Module. In the Management Module Web interface,<br />
select MM Control → Restart MM. <strong>The</strong>n click the Restart button on the<br />
displayed form. You are prompted to confirm the restart before it occurs.<br />
8. Remove the Management Module from the isolated private Ethernet network<br />
and connect it to the network you will use to manage the <strong>BladeCenter</strong>.<br />
You can now connect to the Management Module Web and command line<br />
interfaces using the IP address that you assigned to the Management Module<br />
external network interface.<br />
Now consider performing other Management Module setup tasks, such as:<br />
► Setting the Management Module date and time so that log entries have useful<br />
timestamps<br />
► Defining user IDs and passwords for the system administrators and operators<br />
who will manage the <strong>BladeCenter</strong><br />
Alternatively, you can configure the Management Module to use a Lightweight<br />
Directory Access Protocol (LDAP) directory for this purpose.<br />
► Configuring the Management Module to send alerts to management systems<br />
via SNMP, or system administrators using e-mail via Simple Mail Transfer<br />
Protocol (SMTP)<br />
► Enabling the use of Secure Sockets Layer (SSL) to securely access the<br />
Management Module Web interface<br />
► Enabling the use of Secure Shell (SSH) to securely access the Management<br />
Module command line interface (CLI)<br />
For additional information about how to perform these tasks, refer to the<br />
<strong>IBM</strong> Eserver <strong>BladeCenter</strong> Management Module User’s Guide.<br />
5.2 LAN Switch I/O Module configuration<br />
This section explains the basic set up LAN Switch I/O Modules so you can work<br />
with <strong>BladeCenter</strong> <strong>JS20</strong>s. Advanced configuration of LAN Switch I/O Modules is<br />
beyond the scope of this book.<br />
<strong>The</strong> primary setup tasks for LAN Switch I/O Modules are to:<br />
► Assign IP addresses to the LAN Switch I/O Module management interfaces<br />
► Enable the LAN Switch I/O Module external Ethernet ports<br />
You can learn about the selection of IP addresses for the LAN Switch I/O Module<br />
management interfaces in Chapter 4, “Planning considerations” on page 53.<br />
Chapter 5. Hardware setup 73
When a new LAN Switch I/O Module is first installed, the Management Module<br />
assigns a default IP address to the management interface of the LAN Switch I/O<br />
Module. <strong>The</strong> default IP address is chosen based on the I/O module bay where<br />
the LAN Switch I/O Module is installed. Those I/O modules installed in I/O<br />
module bays 1, 2, 3, and 4 are assigned IP addresses 192.168.70.127,<br />
192.168.70.128, 192.168.70.129, and 192.168.70.130, respectively.<br />
Set the IP address of each LAN Switch I/O Module management interface and<br />
enable the external ports:<br />
1. Connect to the Management Module using the Web browser interface as<br />
explained in “Configuring the Management Module” on page 71.<br />
2. From the Management Module Web interface, select I/O Module Tasks →<br />
Management.<br />
3. In the form shown in Figure 5-3, scroll down to the entry for the LAN Switch<br />
I/O Module that you want to configure. Enter the IP address, subnet mask,<br />
and gateway that you want to assign to the LAN Switch I/O Module<br />
management interface. Click the Save button to activate the new IP address.<br />
Figure 5-3 LAN Switch I/O Module IP address<br />
74 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
4. You are prompted to confirm that you want to change the IP address.<br />
5. Select the Advanced Management link for the LAN Switch I/O Module.<br />
6. Scroll down to the Advanced Setup section of the displayed form. This is<br />
illustrated in Figure 5-4. For the External Ports field, select Enabled from the<br />
drop-down list and then click the Save button.<br />
Figure 5-4 LAN Switch I/O Module setup<br />
Chapter 5. Hardware setup 75
At this point, the LAN Switch I/O Module management interface has an IP<br />
address. Also, the external ports on the LAN switch module are enabled so that<br />
they can be used to communicate with blade servers.<br />
<strong>The</strong> SoL remote text console function of the Management Module depends on a<br />
VLAN provided by a LAN Switch I/O Module installed in I/O module bay 1. This<br />
VLAN is automatically provided by the 4-Port Gigabit Ethernet Switch Module<br />
and the Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module using VLAN<br />
4095.<br />
If you are using a Cisco Systems Intelligent Gigabit Ethernet Switch Module in<br />
I/O module bay 1, you must manually configure this VLAN. A procedure is<br />
documented in the Cisco Systems Intelligent Gigabit Ethernet Switch Module for<br />
the <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Installation Guide. This Cisco guide describes<br />
how to manually set up the VLAN that is needed to support the SoL remote text<br />
console function.<br />
5.3 Blade server configuration<br />
Minimal configuration is required for each <strong>BladeCenter</strong> <strong>JS20</strong> prior to installing an<br />
operating system. <strong>The</strong> two main configuration tasks are:<br />
► Assigning a name to the blade server<br />
► Setting the blade server boot sequence<br />
<strong>The</strong> easiest way to perform these tasks is through the Management Module Web<br />
interface.<br />
76 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
5.3.1 Assigning names to blade servers<br />
Set the name of each blade server in a <strong>BladeCenter</strong> chassis by using these<br />
steps:<br />
1. From the Management Module Web interface, select Blade Tasks →<br />
Configuration.<br />
2. In the Blade Information section of the form, as shown in Figure 5-5, type the<br />
name that you want to assign to each blade server.<br />
3. Scroll down the form and select the Save button to save the names that you<br />
have assigned to each blade server.<br />
Figure 5-5 Blade server naming<br />
Chapter 5. Hardware setup 77
5.3.2 Setting the boot sequence<br />
Set the boot sequence of each blade server by using this procedure:<br />
1. From the Management Module Web interface, select Blade Tasks →<br />
Configuration.<br />
2. Scroll down the form to the Boot Sequence section to see the current boot<br />
sequence for all the blade servers.<br />
3. If a blade server does not have the correct boot sequence, you can change<br />
the boot sequence by selecting the blade server name, which displays the<br />
form shown in Figure 5-6.<br />
4. Set the desired boot sequence using the drop-down lists and click the Save<br />
button to set the new boot sequence.<br />
<strong>The</strong> correct boot sequence depends on the method that you plan to use to install<br />
an operating system on the blade server.<br />
Figure 5-6 Boot sequence of blades<br />
78 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
5.4 Firmware<br />
You may need to upgrade various firmware components before you install an<br />
operating system on a <strong>BladeCenter</strong> <strong>JS20</strong>. <strong>The</strong> minimum firmware levels required<br />
and the actual firmware levels we used during the development of this book are<br />
listed in Table 5-1.<br />
To check the level of firmware components, use the following two steps:<br />
1. Find the latest firmware levels and update packages distributed on the <strong>IBM</strong><br />
support Web site.<br />
a. Go to the <strong>IBM</strong> support Web site:<br />
http://www.ibm.com/pc/support<br />
b. Under Download, select Driver matrices.<br />
c. On the Personal computing drivers Web page, select Servers.<br />
d. On the Software and device drivers - Servers Web page, select <strong>eServer</strong><br />
<strong>BladeCenter</strong>.<br />
e. On the “Software and device drivers - <strong>IBM</strong> <strong>eServer</strong> <strong>BladeCenter</strong> (Type<br />
8677)” Web page, you see the firmware levels for the <strong>BladeCenter</strong>.<br />
f. You can also select <strong>IBM</strong> <strong>eServer</strong> <strong>BladeCenter</strong> <strong>JS20</strong> - Software and<br />
device drivers to see <strong>JS20</strong>-specific firmware levels.<br />
2. Identify the current build levels and levels of firmware that are currently<br />
installed by displaying the firmware vital product data (VPD). You can view the<br />
firmware VPD for most firmware components, as shown in Table 5-1, through<br />
the Management Module Web interface. This is illustrated in Figure 5-7.<br />
Table 5-1 <strong>BladeCenter</strong> and <strong>BladeCenter</strong> <strong>JS20</strong> firmware components<br />
Firmware component Minimum level Level we used<br />
Management Module V1.09 (BRET58A) V1.14 (BRET67I)<br />
4-Port Gigabit Ethernet<br />
Switch Module<br />
V1.04 (ibmrun.081) V1.07 (ibmrun.095)<br />
<strong>BladeCenter</strong> <strong>JS20</strong> BIOS FW04310120 FW04310120<br />
<strong>BladeCenter</strong> <strong>JS20</strong> BSMP V1.00 (BQ8T16A) V1.0 (BQ8T20A)<br />
Note: FW04310120 for the <strong>JS20</strong> BIOS is the minimum required supported<br />
level to support AIX 5L releases like V5.2 (ML4).<br />
Chapter 5. Hardware setup 79
Figure 5-7 Viewing firmware levels on <strong>BladeCenter</strong><br />
<strong>The</strong> following sections explain how to update the various firmware components<br />
associated with the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
5.4.1 Management Module firmware<br />
<strong>The</strong> easiest way to upgrade the firmware of the Management Module is through<br />
the Management Module Web interface.<br />
1. Download the firmware update package from the <strong>IBM</strong> support Web site. This<br />
is usually a ZIP file that contains several files with a PKT extension.<br />
80 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
2. Unzip the firmware update package into a directory on the workstation where<br />
you are running the Web browser that you are using to connect the<br />
Management Module Web interface.<br />
3. Update each PKT file to the Management Module as explained in the<br />
following steps.<br />
a. In the navigation panel of the Management Module Web interface<br />
(Figure 5-8), select MM Control → Firmware Update.<br />
b. In the Update MM Firmware pane (right side of Figure 5-8), select the<br />
Browse button and locate the PKT file in the directory where you<br />
unpacked it.<br />
c. Click the Update button to initiate the download of the PKT file to the<br />
Management Module. This step may take some time to complete.<br />
Figure 5-8 Updating the Management Module firmware<br />
d. A window opens that shows you the progress of the download.<br />
e. Click the Continue button when prompted to confirm the firmware update.<br />
This step may also take some time to complete.<br />
f. Another window opens that shows you the progress.<br />
Chapter 5. Hardware setup 81
4. After you download all the PKT files to the Management Module, restart the<br />
Management Module to make the firmware updates active. In the<br />
Management Module Web interface, select MM Control → Restart MM.<br />
5.4.2 LAN Switch I/O Module firmware<br />
<strong>The</strong> method you should use to upgrade the firmware of a LAN switch module<br />
depends on the type of switch module. We demonstrate how you can upgrade<br />
firmware for the 4-port Gigabit Ethernet Switch Module, which is the switch<br />
module we used in our testing.<br />
For information about upgrading firmware in the Nortel Networks Layer 2-7<br />
Gigabit Ethernet Switch Module, refer to the Nortel Networks Layer 2-7 GbE<br />
Switch Module for <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Installation Guide.<br />
For information about upgrading firmware in the Cisco Systems Intelligent<br />
Gigabit Ethernet Switch Module, refer to the Cisco Systems Intelligent Gigabit<br />
Ethernet Switch Module for the <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Installation Guide.<br />
1. Download the firmware update package from the <strong>IBM</strong> support Web site. This<br />
is usually a ZIP file that contains the firmware image file.<br />
2. Unzip the firmware update package into a directory on the workstation where<br />
you are running the Web browser that you use to connect the Management<br />
Module Web interface.<br />
3. Before you can upgrade the firmware of the 4-Port Gigabit Ethernet Switch<br />
Module, configure an IP address for the switch module management<br />
interface. This enables the capability to directly connect to the switch module<br />
Web interface. To configure the IP address of the switch module management<br />
interface, see 5.2, “LAN Switch I/O Module configuration” on page 73.<br />
4. Connect to the switch module Web interface via the Management Module<br />
Web interface using the following procedure:<br />
a. In the Management Module Web interface, select I/O Module Tasks →<br />
Management.<br />
b. Select the Advanced Management link for the I/O module bay where the<br />
switch module is installed.<br />
c. Scroll down to the bottom of the page to the Start Telnet/Web Session<br />
category. Click the Start Web Session button.<br />
d. A new window opens for the switch module Web interface. You are<br />
prompted to provide a user ID and a password. <strong>The</strong> default user ID is<br />
USERID, and the default password is PASSW0RD, where the number 0 is<br />
between the letters W and R in the password. Consider changing the<br />
defaults in a production environment.<br />
82 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
You can also connect directly to the switch module user interface from any<br />
Web browser if you know the IP address of the switch module.<br />
5. Upgrade the firmware using the following procedure:<br />
a. From the switch module Web interface, select Maintenance → Using<br />
Browser → Upgrade Firmware/Configuration File.<br />
b. You now see the form shown in Figure 5-9. Select the Browse button and<br />
locate the switch module firmware file in the directory where you put it<br />
earlier.<br />
c. Select the Start button to initiate the switch module firmware update<br />
process.<br />
d. A window opens requesting confirmation. After you confirm the update,<br />
the main window displays the status of the update until it completes and<br />
reboots the switch module.<br />
Figure 5-9 Firmware upgrade of 4-Port Gb Ethernet Switch Module<br />
6. After you restart the switch module, verify that the IP address of the switch<br />
module is still set correctly through the Management Module Web interface. In<br />
some cases, you may need to reset the IP address.<br />
Chapter 5. Hardware setup 83
5.4.3 <strong>BladeCenter</strong> <strong>JS20</strong> firmware (BIOS)<br />
<strong>The</strong>re is currently no way to upgrade the firmware (BIOS) of a <strong>BladeCenter</strong> <strong>JS20</strong><br />
without first installing an operating system on the blade server. This is usually not<br />
an issue, because newly shipped <strong>BladeCenter</strong> <strong>JS20</strong>s already have current<br />
firmware installed.<br />
For completeness, we explain how to upgrade the firmware of a <strong>BladeCenter</strong><br />
<strong>JS20</strong> after you install an operating system.<br />
Upgrading firmware under AIX<br />
AIX includes the update_flash utility as part of the AIX diagnostics package. To<br />
use the update_flash utility, you must first download the latest firmware from the<br />
<strong>IBM</strong> support Web site. <strong>The</strong>n transfer it to a disk that is accessible to the operating<br />
system running on the <strong>BladeCenter</strong> <strong>JS20</strong>. Next, you start the update_flash utility<br />
from the directory where the new firmware is located. <strong>The</strong> update_flash utility<br />
requires that you have root privileges.<br />
<strong>The</strong> following example illustrates the usage of update_flash:<br />
/usr/lpp/diagnostics/bin/update_flash -f JS1FW419A.IMG<br />
In this example, JS1FW419A.IMG is the firmware image that was previously<br />
downloaded from the <strong>IBM</strong> support Web site. <strong>The</strong> update_flash utility asks you to<br />
confirm that you want to update the firmware and then reboot the operating<br />
system to perform the actual update.<br />
If you subsequently have a problem with the new firmware and want to revert to<br />
the previous firmware level, use the following command:<br />
/usr/lpp/diagnostics/bin/update_flash -r<br />
Alternatively, if you are satisfied with the new firmware, commit the firmware<br />
update before you install any future firmware by using this command:<br />
/usr/lpp/diagnostics/bin/update_flash -c<br />
Upgrading firmware under Linux<br />
<strong>The</strong> mechanism used to update the firmware (BIOS) of the <strong>BladeCenter</strong> <strong>JS20</strong><br />
under the Linux operating system is based on a Linux kernel module called<br />
rtas_flash. This kernel module is available in most recent 2.4 kernels and in all<br />
2.6 kernels. All supported Linux distributions for the <strong>BladeCenter</strong> <strong>JS20</strong> include<br />
this kernel module.<br />
<strong>The</strong> rtas_flash kernel module creates interfaces for manipulating the flash<br />
memory where the firmware is stored in the /proc/ppc64/rtas directory. In most<br />
Linux distributions, this kernel module is normally not loaded at system boot.<br />
84 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
<strong>The</strong>refore, the interfaces to manipulate the flash memory are not normally<br />
present in /proc/ppc64/rtas.<br />
<strong>IBM</strong> provides a script for Linux called update_flash that uses the interfaces<br />
provided by the rtas_flash kernel module. It also supports the same options and<br />
syntax as the update_flash utility under AIX.<br />
You can obtain the latest version of the update_flash script by installing the<br />
diagnostic utilities for Linux on POWER that <strong>IBM</strong> distributes from the Web at:<br />
http://techsupport.services.ibm.com/server/lopdiags<br />
Note: Some Linux distributions provide a version of the update_flash script<br />
that is not fully functional. Always obtain the latest version of the update_flash<br />
script from <strong>IBM</strong> before you attempt to update firmware.<br />
To use the update_flash script, follow these steps:<br />
1. Download the latest firmware from the <strong>IBM</strong> support Web site.<br />
2. Transfer it to a disk that is accessible to the operating system running on the<br />
<strong>BladeCenter</strong> <strong>JS20</strong>.<br />
3. Enter the update_flash command from the directory where the new firmware<br />
is located. <strong>The</strong> update_flash command requires that you have root privileges.<br />
<strong>The</strong> following example illustrates the usage of update_flash:<br />
/usr/sbin/update_flash -f JS1FW419A.IMG<br />
In this example, JS1FW419A.IMG is the firmware image that was previously<br />
downloaded from the <strong>IBM</strong> support Web site. <strong>The</strong> update_flash utility copies the<br />
new firmware image into the kernel and then reboots the operating system to<br />
perform the actual firmware update.<br />
If you subsequently have a problem with the new firmware and want to revert to<br />
the previous firmware level, then use the following command:<br />
/usr/sbin/update_flash -r<br />
If you are happy with the new firmware, commit the firmware upgrade before you<br />
install any future firmware by using the following command:<br />
/usr/sbin/update_flash -c<br />
5.4.4 Integrated Systems Management Processor firmware<br />
<strong>The</strong> easiest way to upgrade the firmware of the <strong>BladeCenter</strong> <strong>JS20</strong> Integrated<br />
Systems Management Processor (ISMP) is through the Management Module<br />
Web interface.<br />
Chapter 5. Hardware setup 85
1. Download the firmware update package from the <strong>IBM</strong> support Web site. This<br />
package contains a file with a ZIP extension, which is the firmware for the<br />
ISMP.<br />
2. Place the file into a directory on the workstation where you will run the Web<br />
browser that you will use to connect the Management Module Web interface.<br />
3. Download the PKT file to the ISMP. Follow this upgrade procedure:<br />
a. As shown in Figure 5-10, select Blade Tasks → Firmware Update in the<br />
Management Module Web interface.<br />
b. <strong>The</strong> Update Blade Firmware pane is displayed (right side of Figure 5-10).<br />
In the Target drop-down list, select the blade server on which you want to<br />
upgrade the ISMPfirmware.<br />
c. Click the Browse button and locate the ISMP firmware ZIP file in the<br />
directory where you placed it earlier.<br />
d. Click the Update button to initiate download of the ZIP file to the<br />
Management Module. This step may take some time to complete.<br />
Figure 5-10 Blade firmware update<br />
86 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
e. A window opens that shows the progress of the download.<br />
f. Click the Continue button when prompted to confirm the firmware update.<br />
This step may also take some time to complete.<br />
g. A window opens that shows the progress.<br />
<strong>The</strong> new ISMP firmware is now active in the target <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
5.5 Providing a console for the <strong>JS20</strong> blades<br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> differs from other blade servers that are available for the<br />
<strong>BladeCenter</strong> in that it does not provide an interface to a keyboard, video monitor,<br />
and mouse (KVM) console. <strong>The</strong>refore, you must set up the SoL remote text<br />
console function to provide a console for a <strong>BladeCenter</strong> <strong>JS20</strong>. Use of SoL is<br />
optional for other types of blade server that support KVM consoles.<br />
<strong>The</strong> SoL remote text console function involves several different components in<br />
the <strong>BladeCenter</strong> infrastructure as illustrated in Figure 5-11.<br />
<strong>The</strong> SoL remote console function works as follows:<br />
1. Using a Telnet or SSH client, connect to the <strong>BladeCenter</strong> Management<br />
Module CLI. This is usually via an external management network that is<br />
connected to the Management Module’s 10/100BaseT Ethernet interface.<br />
2. From the Management Module’s CLI, initiate a SoL remote console session to<br />
the desired blade server.<br />
3. <strong>The</strong> Management Module uses a private VLAN provided by the LAN switch<br />
module in I/O module bay 1, to transport the SoL data stream to the Ethernet<br />
interface of the target blade server.<br />
4. <strong>The</strong> Ethernet controller of the target blade server passes the SoL data stream<br />
received from the private VLAN to the blade system management processor<br />
(BSMP), which manages the text console for the blade server.<br />
Chapter 5. Hardware setup 87
LAN Switch<br />
Module in<br />
I/O Module<br />
Bay 1<br />
BSMP<br />
Figure 5-11 Serial over LAN components<br />
<strong>The</strong> sections that follow explain how to configure and use the SoL remote text<br />
console function.<br />
88 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
Telnet<br />
or<br />
SSH Client<br />
External Management Network<br />
Ethernet<br />
Interface<br />
(eth0)<br />
Management<br />
Module<br />
External Interface<br />
(eth0)<br />
Internal Interface<br />
(eth1)<br />
Ethernet<br />
Interface<br />
(eth1)<br />
Ethernet Controller<br />
<strong>JS20</strong> Blade Server<br />
I/O Module<br />
Bay 2
5.5.1 Configuring Serial over LAN<br />
Before you attempt to configure the SoL remote text console function, verify that<br />
you have all the prerequisites in place:<br />
► A supported LAN switch module installed in I/O module bay 1<br />
This switch module is used to provide a VLAN that connects the Management<br />
Module to the first Ethernet interface on each blade server.<br />
► <strong>The</strong> minimum firmware levels described in 5.4, “Firmware” on page 79<br />
This is important if you install a <strong>BladeCenter</strong> <strong>JS20</strong> in an existing <strong>BladeCenter</strong><br />
chassis that may have back-level firmware in the Management Module or LAN<br />
switch modules.<br />
► A reliable Telnet or SSH client<br />
► A network connecting the Telnet or SSH client to the <strong>BladeCenter</strong><br />
Management Module external 10/100BaseT Ethernet interface<br />
► An identified range of IP addresses that will be used by the Management<br />
Module to communicate with the BSMP on each blade server via the private<br />
VLAN<br />
This was discussed in Chapter 4, “Planning considerations” on page 53.<br />
<strong>The</strong> easiest way to configure the SoL remote text console function is through the<br />
Management Module Web interface as explained in the following steps.<br />
To configure the SoL remote text console function, use the following procedure:<br />
1. From the Management Module Web interface, select Blade Tasks → Serial<br />
Over LAN.<br />
2. In the right pane, scroll down until you see the Serial Over LAN Configuration<br />
section (see Figure 5-12). Complete the following tasks:<br />
a. From the Serial over LAN list, select the Enabled option.<br />
b. Leave the value for SoL VLAN ID at the default (4095) if you have either a<br />
4-port Gigabit Ethernet Switch Module or a Nortel Networks Layer 2-7<br />
Gigabit Ethernet Switch Module installed in I/O module bay 1.<br />
If you have a Cisco Systems Intelligent Gigabit Ethernet Switch Module<br />
installed in I/O module bay 1, set the VLAN ID to the same value you used<br />
when you manually defined the VLAN that supports SoL.<br />
c. Enter the start of the IP address range that will be used by the<br />
Management Module to communicate with the BSMP on each blade<br />
server.<br />
d. Leave the values for Accumulate timeout, Send threshold, Retry count,<br />
and Retry interval at their defaults (5, 250, 3, and 250).<br />
Chapter 5. Hardware setup 89
e. Leave the values for User Defined Keystroke Sequences at their defaults.<br />
f. Click the Save button.<br />
Figure 5-12 Serial over LAN configuration<br />
3. Restart the Management Module.<br />
a. In the Management Module Web interface, select MM Control → Restart<br />
MM.<br />
b. Click the Restart button on the displayed form.<br />
c. You are prompted to confirm the restart before it occurs.<br />
4. After the Management Module has restarted and you have reconnected to the<br />
Management Module Web interface from your Web browser, enable the SoL<br />
remote text console for each blade server.<br />
a. Select Blade Tasks → Serial Over LAN from the Management Module<br />
Web interface.<br />
b. In the right pane, scroll down until you see the Serial Over LAN Status<br />
section (see Figure 5-13).<br />
c. Select the blade servers you want to enable for SoL. You can choose all of<br />
them by clicking the check box at the top of the table.<br />
90 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
d. Click the Enable Serial Over LAN link underneath the table. You may<br />
need to scroll down to see this link.<br />
Figure 5-13 Serial over LAN status<br />
e. After a few seconds, the window refreshes. In the SoL column of the table,<br />
verify that each blade server has a status of Enable.<br />
<strong>The</strong> configuration of the SoL remote text console function is now complete.<br />
5.5.2 Using Serial over LAN<br />
Access to the SoL remote text console function is via the Management Module<br />
CLI. <strong>The</strong> CLI is documented in the <strong>IBM</strong> ^ <strong>BladeCenter</strong> Management<br />
Module Command-Line Interface Reference Guide.<br />
You can connect to the Management Module CLI using either a Telnet or SSH<br />
client. Figure 5-14 shows an active Telnet session that displays the help<br />
command. <strong>The</strong> Management Module can support up to 20 concurrent CLI<br />
Chapter 5. Hardware setup 91
connections. This is sufficient to support the concurrent use of a SoL remote text<br />
console to each blade server in a full <strong>BladeCenter</strong> chassis. At the same time, it<br />
supports six additional CLI connections for other administrative activities.<br />
Figure 5-14 Management Module’s command line interface<br />
Restriction: You can only have one active SoL remote text console<br />
connection to each blade server.<br />
<strong>The</strong> Management Module CLI is context sensitive. When you issue CLI<br />
commands, you can accept the current context or override the context using the<br />
-T option provided on most commands. In our examples, we use the -T option to<br />
make it clear on which entity the command is operating. You can change the<br />
context for the Management Module CLI by using the env -T command.<br />
92 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
To use the SoL remote text console function, first connect to the Management<br />
Module from a Telnet or SSH client. You are prompted to enter a user ID and<br />
password. <strong>The</strong> default user ID is USERID and the default password is<br />
PASSW0RD, where 0 in the default password is a zero. Consider changing the<br />
defaults in a production environment.<br />
<strong>The</strong>re are several different commands that you can use to access the SoL<br />
remote text console function. <strong>The</strong> most useful commands are listed in Table 5-2.<br />
Table 5-2 Management Module commands for SoL<br />
Command What it does<br />
console Open a SoL remote text console for the blade server. This<br />
command fails if another SoL remote text console is already open<br />
for the blade server.<br />
console -o Terminate any existing SoL remote text console for the blade server<br />
and open a SoL remote text console for the blade server.<br />
boot -c Reset the blade server, and then open a SoL remote text console<br />
for the blade server.<br />
reset -c This is functionally equivalent to the boot -c command when used<br />
in a blade server context.<br />
power -on -c Power on the blade server and then open a SoL remote text<br />
console for the blade server.<br />
power -cycle -c Power on the blade server and then open a SoL remote text<br />
console for the blade server. If the blade server is already powered<br />
on, power it off first, and then power on.<br />
Here are some examples of how to use these commands. To open a SoL remote<br />
text console to the blade server in bay 3, you use the following command:<br />
console -T system:blade[3]<br />
To reset the blade server in bay 2 and then open a SoL remote text console to the<br />
blade server, you use the following command:<br />
boot -c -T system:blade[2]<br />
To terminate an active SoL remote text console, press the Esc key followed by an<br />
open parenthesis (Shift+9 on U.S. keyboards). When the SoL remote text<br />
console ends, you return to the Management Module CLI prompt.<br />
Inactivity timeout<br />
<strong>The</strong> Management Module CLI has an aggressive default inactivity timeout. If you<br />
do not use the CLI or a SoL remote text console accessed through the CLI for 2<br />
Chapter 5. Hardware setup 93
minutes, you are disconnected. You can increase the timeout using the<br />
telnetcfg command. For example, the following command increases the<br />
inactivity timeout to 10 minutes (600 seconds):<br />
telnetcfg -t 600 -T system:mm[1]<br />
Setting CRLF<br />
Over the years, there have been many variations on the interpretation of the<br />
Enter key in a Telnet session. Some Telnet daemons expect carriage return +<br />
linefeed (CRLF) and some expect carriage return alone (CR). From our testing,<br />
CR seems to be the most compatible. During your SoL session, if when you<br />
press the Enter key, the key behaves as though it was pressed twice and not<br />
once, then correct the CRLF option. Under Windows 2000 Telnet, do this as<br />
shown in Figure 5-15 before you open a connection to the Management Module.<br />
Microsoft (R) Windows 2000 (TM) Version 5.00 (Build 2195)<br />
Welcome to Microsoft Telnet Client<br />
Telnet Client Build 5.00.99206.1<br />
Escape Character is 'CTRL+]'<br />
Microsoft Telnet> unset crlf<br />
Microsoft Telnet> open bcmm<br />
Figure 5-15 Setting CRLF in Windows 2000 Telnet<br />
Refer to the developer of your client for other Telnet implementations.<br />
5.5.3 Open Firmware interfaces<br />
In some situations, you may need to access the Open Firmware interfaces to the<br />
<strong>BladeCenter</strong> <strong>JS20</strong> firmware. This section explains how to access Open Firmware<br />
interfaces and how to use some of the functions they provide.<br />
Activating the Open Firmware interfaces<br />
You can access the Open Firmware interface whenever you power on a<br />
<strong>BladeCenter</strong> <strong>JS20</strong>. Before you can access the Open Firmware interface, set up<br />
the SoL remote text console function as described in 5.5, “Providing a console for<br />
the <strong>JS20</strong> blades” on page 87.<br />
Use the following procedure to access the Open Firmware interface:<br />
1. Using a Telnet or SSH client, connect to the Management Module external<br />
Ethernet interface IP address.<br />
94 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
2. When prompted, enter a valid user ID and password. <strong>The</strong> default<br />
Management Module user ID is USERID, and the default password is<br />
PASSW0RD, where the 0 in the default password is a zero. Consider<br />
changing the defaults in any production environment.<br />
3. Power cycle the blade and start an SoL console by using the power -cycle -c<br />
command. For example, to power cycle and start a SoL remote text console<br />
with the blade server in the first bay (bay 1), use the command:<br />
power -cycle -c -T system:blade[1]<br />
4. After approximately 30 seconds, you see a sequence of checkpoint codes<br />
displayed on the console. <strong>The</strong>se codes are generated by the Power On Self<br />
Test (POST). <strong>The</strong> meaning of the checkpoint codes is documented in the<br />
<strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong> Type 8842 Installation and User’s Guide.<br />
5. Wait until the checkpoint code D5BB is displayed and then type 8. <strong>The</strong> POST<br />
pauses for approximately five seconds at this checkpoint code to give you<br />
time to respond to the checkpoint code. A number is displayed after the D5BB<br />
checkpoint code, indicating how many seconds remain before the POST will<br />
proceed further. For example, Figure 5-16 shows that four seconds remain.<br />
E153 U8842.P1Z.23A1073-P1-T7<br />
EAA1 U8842.P1Z.23A1073-P1<br />
E172 U8842.P1Z.23A1073-P1<br />
EAA1 U8842.P1Z.23A1073-P1<br />
E152 U8842.P1Z.23A1073-P1<br />
E153 U8842.P1Z.23A1073-P1<br />
E152 U8842.P1Z.23A1073-P1<br />
E153 U8842.P1Z.23A1073-P1<br />
EAA1 U8842.P1Z.23A1073-P1<br />
E172 U8842.P1Z.23A1073-P1<br />
EAA1 U8842.P1Z.23A1073-P1-C5<br />
D001<br />
D003<br />
D004<br />
E139<br />
E14A<br />
D008<br />
E1F0<br />
E1F1<br />
D099<br />
D5BB 4<br />
Figure 5-16 POST checkpoint codes<br />
Chapter 5. Hardware setup 95
6. As soon as you type 8, the Open Firmware command prompt is displayed.<br />
If you do not type any key when the D5BB checkpoint code is displayed, the<br />
blade server proceeds to boot using the default boot sequence. To learn how to<br />
configure the boot sequence, see 5.3, “Blade server configuration” on page 76.<br />
5.5.4 Specifying IP parameters to Open Firmware<br />
You can use a directed bootp to install a <strong>JS20</strong> blade from a NIM server. bootp<br />
does not require the NIM server to be on the same subnet as the <strong>JS20</strong> blade.<br />
Nor does this option require that you have the MAC address of the network<br />
adapter on the <strong>JS20</strong> blade.<br />
To perform a directed bootp, you need an SoL connection to the blade so that<br />
you can specify the IP parameters to Open Firmware. Currently you must have<br />
two network adapters to perform a NIM installation if you are using SoL. You<br />
cannot install AIX 5L over the same adapter that is using SoL.<br />
Step 1: Preparing your NIM server<br />
Follow these steps to prepare your NIM server.<br />
1. Create a SPOT, lpp_source, and any other resources that you need at the<br />
level of AIX 5L that you want to install on your NIM server. Your NIM server is<br />
usually the NIM master, but you can also set up a NIM client as a NIM server.<br />
For instructions on how to create NIM resources, see 7.1, “Minimal NIM<br />
installation” on page 124.<br />
2. Ensure that you have the information in the following worksheet (Table 5-3) for<br />
your <strong>JS20</strong> blade before you proceed with the installation.<br />
Table 5-3 Network configuration information worksheet<br />
Network attribute Value<br />
Network interface<br />
Host name<br />
IP address _______.________.________.________<br />
Network mask _______.________.________.________<br />
Name server _______.________.________.________<br />
Domain name _______.________.________.________<br />
Gateway _______.________.________.________<br />
96 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
3. Define the <strong>JS20</strong> blade as a NIM client on your NIM master by running the<br />
following command on the NIM master:<br />
smitty nim_mkmac<br />
This command creates a client definition for your <strong>JS20</strong> blade. You can also<br />
define the <strong>JS20</strong> blade using the define NIM operation on the command line.<br />
4. If you want to set the <strong>JS20</strong> blade's name server and domain name after the<br />
installation, use a resolv_conf resource.<br />
5. Set up your NIM master to install the <strong>JS20</strong> blade, by running this command:<br />
smitty nim_bosinst<br />
Select the <strong>JS20</strong> blade that you defined earlier as your target. <strong>The</strong>n select the<br />
type of install that you want to perform and select the installation resources<br />
that you want to use to install the <strong>JS20</strong> blade. You can also prepare the <strong>JS20</strong><br />
blade to install using the bos_inst NIM operation on the command line.<br />
Note: If the <strong>JS20</strong> blade is powered off or was never installed, set “Initiate<br />
reboot and installation now?” to no and press Enter in the SMIT interface.<br />
If the <strong>JS20</strong> blade is powered on and running AIX 5L, set “Initiate reboot<br />
and installation now?” to yes in the SMIT interface. If you choose this<br />
option, a directed bootp is initiated by default and you can skip step 2.<br />
Before you run this command, ensure that the <strong>JS20</strong> blade is a registered<br />
NIM client.<br />
1. Run smitty niminit on the <strong>JS20</strong> blade.<br />
2. Specify the host name of your NIM master and the interface you want to<br />
use for the installation.<br />
You can also initialize the <strong>JS20</strong> blade using the niminit command on the<br />
command line.<br />
Step 2: Specifying a directed bootp from the <strong>JS20</strong> blade<br />
Specify a directed bootp from the <strong>JS20</strong> blade by using these steps:<br />
1. Open a Web interface to the Management Module by navigating to the IP<br />
address or host name of the Management Module using a Web browser.<br />
2. Enable SoL to the <strong>JS20</strong> blade from the Management Module Web interface.<br />
Select Blade Tasks →Serial Over LAN.<br />
3. Select the <strong>JS20</strong> blade that you are installing and click Enable Serial Over<br />
LAN.<br />
4. Power on the <strong>JS20</strong> blade from the Management Module Web interface Blade<br />
Tasks →Power/Restart.<br />
Chapter 5. Hardware setup 97
5. Select the <strong>JS20</strong> blade that you are installing and click Power On Blade.<br />
6. Open a Serial over LAN connection to the <strong>JS20</strong> blade by using Telnet into the<br />
Management Module and running the console command. For example, if the<br />
<strong>JS20</strong> blade is in slot 3, run the following command:<br />
console -T blade[3]<br />
<strong>The</strong> Serial over LAN connection shows a series of LED numbers.<br />
7. Type 8 when you see D5BB to go to the Open Firmware prompt.<br />
8. Boot from the network by typing:<br />
boot net:bootp,server_ip,,client_ip,gateway_ip<br />
Because SoL uses ent0, you must use ent1. Run a command similar to this<br />
one (note the double comma):<br />
boot<br />
/pci@8000000f8000000/pci@0/ethernet@1,1:bootp,192.168.2.10,,192.168.1.11,192.168.1.1<br />
98 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
Note: You must specify the full device path name with this command. To<br />
determine the full path to your device, list the device tree by running the ls<br />
command at the Open Firmware prompt. This command displays output<br />
similar to this:<br />
0 > ls<br />
000000c87f18: /ibm,serial<br />
000000c88840: /chosen<br />
000000c88a98: /packages<br />
...<br />
000000d31488: /vdevice<br />
000000d327a8: /vty@0<br />
000000d32f88: /<strong>IBM</strong>,sp@4000<br />
000000d33f10: /rtc@4001<br />
000000d34a18: /pci@8000000f8000000<br />
000000d384d0: /pci@0<br />
000000d4bbd0: /ethernet@1<br />
000000d5af50: /ethernet@1,1<br />
000000d3be00: /pci@3<br />
000000d6a350: /usb@0<br />
000000d845f8: /hub@1<br />
000000d854b8: /usb@0,1<br />
000000d9f760: /hub@1<br />
000000d3f798: /pci@1f<br />
000000d45ed8: /ide@4,1<br />
000000d47b10: /disk@0<br />
<strong>The</strong> highlighted items are the path to the second Ethernet adapter. You<br />
pass this information to the boot command to initiate a network boot from<br />
the second Ethernet adapter.
After you run the boot command, the network installation begins. Output similar<br />
to Example 5-1 is displayed on the SoL connection.<br />
Example 5-1 Output of the boot command<br />
BOOTP: chosen-network-type =<br />
ethernet,auto,none,auto<br />
BOOTP: server IP = 192.168.2.10<br />
BOOTP: requested filename =<br />
BOOTP: client IP = 192.168.1.11<br />
BOOTP: client HW addr = 0 d 60 1e c cb<br />
BOOTP: gateway IP = 192.168.1.1<br />
BOOTP: device<br />
/pci@8000000f8000000/pci@0/ethernet@1,1<br />
BOOTP: loc-code U8842.P1Z.23A0984-P1-T7<br />
BOOTP R = 1<br />
FILE: /tftpboot/js20blade1.austin.ibm.com<br />
Load Addr=0x0000000000004000, Max<br />
Size=0x0000000000bfc000<br />
FINAL Packet Count = 21131<br />
FINAL File Size = 10818623 bytes.<br />
load-base=0x4000<br />
real-base=0xc00000<br />
Elapsed time since release of system<br />
processors: 2 mins 28 secs<br />
Chapter 5. Hardware setup 99
100 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Chapter 6. Installing Linux<br />
6<br />
This chapter explains how to install Red Hat Enterprise Linux (RHEL) and SUSE<br />
LINUX Enterprise Server (SLES) on the <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 101
6.1 Installing Linux<br />
Both Red Hat Enterprise Linux and SUSE LINUX Enterprise Server are<br />
supported operating systems for the <strong>BladeCenter</strong> <strong>JS20</strong>. This chapter explains<br />
how to set up and conduct a network-based installation of both distributions.<br />
Because the general process of doing this is similar for both operating systems,<br />
the focus of this chapter is on the installation of Red Hat Enterprise Linux 3.0 and<br />
SUSE LINUX Enterprise Server 9.0.<br />
Tip: Prior to installing any operating system, we recommend that you upgrade<br />
all firmware to the latest level. See 5.4, “Firmware” on page 79, for instructions<br />
about how to complete this task.<br />
6.1.1 Configuring the sources<br />
Since we are doing a network installation, the first step is to configure a file space<br />
that contains the source files for the operating system installation. For all cited<br />
examples, we created the sources on a pSeries 690 that was connected to the<br />
same local area network (LAN) as the <strong>BladeCenter</strong>. Since there are special tools<br />
available within each distribution for this purpose, we configured each network<br />
install server with a similar distribution. For example, the server that installed<br />
RHEL 3 also ran RHEL 3.<br />
Tip: We recommend a 100 Mb or faster LAN that is local to both the machine<br />
that will host the files and the <strong>BladeCenter</strong>. We also recommend that the file<br />
spaces be shared via Network File System (NFS).<br />
Red Hat sources<br />
<strong>The</strong> Red Hat installer, Anaconda, makes it simple to set up a remote file share.<br />
Simply copy the ISOs for the source CDs into your NFS mount. To create an ISO<br />
of the first CD and place it in /mnt/exports/, enter:<br />
# dd if=/dev/cdrom of=/mnt/exports/RHEL-3.0-U2-disc1.iso bs=32M<br />
Repeat the previous step for all the CDs, changing the file name as appropriate.<br />
You may also increase the bs parameter as appropriate. This parameter controls<br />
block size. <strong>The</strong> larger the block size is, the more RAM is taken for the dd process,<br />
but the faster the process takes.<br />
Important: Ensure that the CD is not mounted before you begin the dd. Also<br />
ensure that the destination of the ISO has enough space to store all the data.<br />
Copying a full complement of RHEL 3 ISOs can take over 4 GB.<br />
102 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Configure the NFS daemon to export the mount point where the ISOs are<br />
located. You can do this by editing /etc/exports. Example 6-1 shows the export<br />
file we used.<br />
Example 6-1 /etc/exports file contents<br />
/mnt/exports/ 192.168.1.0/24(ro)<br />
After the /etc/exports file is created or updated, we enabled NFS as shown in<br />
Example 6-2.<br />
Example 6-2 Starting NFS<br />
# service nfs start<br />
Starting NFS services: [ OK ]<br />
Starting NFS quotas: [ OK ]<br />
Starting NFS daemon: [ OK ]<br />
Starting NFS mountd: [ OK ]<br />
#<br />
Attention: If NFS does not start properly, refer to the Red Hat Enterprise<br />
Linux Reference Guide at the following Web site for troubleshooting<br />
information:<br />
http://www.redhat.com/docs<br />
Chapter 6. Installing Linux 103
SuSE sources<br />
SuSE installations are usually administered via a program called Yet Another<br />
Setup Tool (YaST).<br />
1. Within YaST, there is an applet for configuring installation sources on the<br />
server. Access this from the Misc section in the left pane, as shown in<br />
Figure 6-1.<br />
Figure 6-1 YaST installation server configuration<br />
104 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
2. You now see the Source Configuration panel (Figure 6-2). Select SuSE SLES<br />
Version 9 (ppc).<br />
Figure 6-2 Selecting the distribution<br />
Chapter 6. Installing Linux 105
3. In the next panel as shown in Figure 6-3, you select which media to use as a<br />
source for the SuSE distribution. We found it easier to use ISOs rather than<br />
CDs. Proceeding past this window builds the source directory. You may be<br />
prompted for additional CDs (or locations of ISOs) while the tree is being built.<br />
Tip: If you have any service packs or additional CDs that you want to add,<br />
select the Prompt for Additional CDs check box.<br />
Figure 6-3 Source Configuration pane<br />
106 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
4. After the source files are copied into the installation share, click Finish as<br />
shown in Figure 6-4. This completes any final tasks such as starting the NFS<br />
daemon.<br />
Figure 6-4 Completing the setup<br />
6.1.2 <strong>The</strong> zImage.initrd file<br />
PowerPC-based hosts support booting directly from the network provided that<br />
they are served a zImage.initrd via BOOTP. <strong>The</strong> zImage.initrd is a package that<br />
contains both the kernel and initial RAM disk in a single file.<br />
Most distributions already have a zImage.initrd, which you can use for network<br />
booting. For RHEL, you can find this file on the first CD as images/netboot.img.<br />
For SuSE, this file is called install and is in the root directory of the first CD.<br />
Although these files are referred to by different names, they are the zImage.initrd<br />
file that we require.<br />
Building your own zImage.initrd<br />
Although most users use the zImage.initrd provided by their distribution, you can<br />
also compile your own.<br />
Tip: This section address the nuances of how to create a zImage.initrd. If you<br />
need more information about compiling a kernel, see the Kernel HOWTO at:<br />
http://www.tldp.org<br />
Chapter 6. Installing Linux 107
To build your own zImage.initrd, follow these steps:<br />
1. Obtain the necessary kernel sources and development utilities for your<br />
distribution as appropriate. If the necessary packages are not installed, you<br />
may experience some errors, which cause your compilation to fail. Consult<br />
your distribution’s documentation to determine the names of these packages.<br />
2. With the packages installed, start by cleaning the source tree. This is a good<br />
procedure to ensure that there are no remnants of an old compile left hanging<br />
around. Here is an example of how to do this.<br />
# make mrproper<br />
3. With the source tree clean, configure your kernel as required. <strong>The</strong> preferred<br />
method to do this is to use either make menuconfig (text based) or make<br />
xconfig (graphical; requires an X server).<br />
Both Red Hat and SuSE have their default kernel options selected when<br />
menuconfig / xconfig is launched after having done a make mrproper. From<br />
either interface, select the desired options.<br />
4. After you configure the kernel, perform the compilation. Since RHEL 3 is<br />
based on the 2.4 kernel and SLES 9 is based on 2.6, the next steps are<br />
slightly different. Refer to Example 6-3 and Example 6-4 for each distribution.<br />
Example 6-3 Compiling a kernel in Red Hat<br />
# make dep<br />
# make clean<br />
# make zImage<br />
# make modules<br />
# make modules_install<br />
# make install<br />
Example 6-4 Compiling a kernel in SuSE<br />
# make<br />
# make modules_install<br />
# make install<br />
108 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
5. After you compile the kernel, create the initial RAM disk. Both Red Hat and<br />
SuSE ship with a command called mkinitrd for this purpose. However, the<br />
syntax for each is slightly different, as shown in Example 6-5 and<br />
Example 6-6.<br />
Example 6-5 mkinitrd for Red Hat<br />
# cd arch/ppc64/boot<br />
# mkinitrd -f ramdisk.image 2.4.21-15.ELcustom<br />
# gzip ramdisk.image<br />
# cd /usr/src/linux-2.4<br />
# make zImage.initrd<br />
Example 6-6 mkinitrd for SuSE<br />
# make install<br />
# mkinitrd<br />
# cd arch/ppc64/boot/<br />
# cp /boot/initrd-2.6.5-7.39-pseries64 ramdisk.image<br />
# gzip ramdisk.image<br />
# cd /usr/src/linux<br />
# make zImage.initrd<br />
Attention: <strong>The</strong> file names referred to are all relative to the kernel version<br />
strings within the Makefile for the kernel. <strong>The</strong> files on your system may be<br />
different.<br />
After you complete these steps, you should have a bootable zImage.initrd file in<br />
/boot/ppc64/.<br />
6.2 Configuring BOOTP and TFTP<br />
<strong>The</strong>re are three steps to network booting a POWER/PowerPC-based server.<br />
1. Perform a Power On Self Test (POST) and then receive BOOTP information<br />
from BOOTP server, including the IP, Trivial File Transfer Protocol (TFTP)<br />
server IP, and zImage.initrd file name.<br />
2. TFTP server sends the zImage.initrd file.<br />
3. <strong>The</strong> server boots using the zImage.initrd file.<br />
Both RHEL and SLES ship with Dynamic Host Configuration Protocol (DHCP)<br />
and TFTP packages. Ensure that you have installed the appropriate RPMs.<br />
Chapter 6. Installing Linux 109
6.2.1 BOOTP<br />
DHCP is an extension to the original BOOTP specification. As a result, you can<br />
use the DHCP daemon (dhcpd) to provide the BOOTP information for booting via<br />
the network.<br />
<strong>The</strong> DHCP daemon is configured through the /etc/dhcpd.conf file. Example 6-7<br />
shows the configuration that we used in our lab.<br />
With dhcpd configured, start the DHCP daemon as shown here:<br />
# /etc/init.d/dhcpd start<br />
Example 6-7 /etc/dhcpd.conf<br />
always-reply-rfc1048 true;<br />
allow bootp;<br />
deny unknown-clients;<br />
not authoritative;<br />
default-lease-time 600;<br />
max-lease-time 7200;<br />
subnet 192.168.1.0 netmask 255.255.255.0 {<br />
group {<br />
next-server 192.168.1.1;<br />
}<br />
Attention: Keep in mind that Example 6-7 contains IP and MAC address<br />
information specific to our lab environment. See your product documentation<br />
or the dhcpd man page to customize this for your site.<br />
}<br />
110 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
host blade1 {<br />
fixed-address 192.168.1.117;<br />
hardware ethernet 00:0d:60:1e:0f:89;<br />
filename "zImage.initrd.suse";<br />
}<br />
host blade2 {<br />
fixed-address 192.168.1.118;<br />
hardware ethernet 00:0d:60:1e:0e:fb;<br />
filename "zImage.initrd.redhat";<br />
}
6.2.2 Trivial File Transfer Protocol<br />
<strong>The</strong> TFTP daemon for both distributions considers the /tftpboot directory its<br />
home. Any file name statements made in the /etc/dhcpd.conf configuration file<br />
should be relative to this path.<br />
Both RHEL and SLES spawn TFTP from the xinetd daemon. By default, tftpboot<br />
is disabled on both distributions. To enable both xinetd and TFTP, enter:<br />
[root@blade1 root]# chkconfig tftp on<br />
[root@blade1 root]# /etc/init.d/xinetd restart<br />
Your installation server is now ready to network boot the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
6.3 Preparing an unattended install<br />
RHEL and SLES both have their own methods of conducting an unattended<br />
installation. RHEL uses Kickstart, and SLES uses a method known as AutoYaST.<br />
Chapter 6. Installing Linux 111
6.3.1 Red Hat Kickstart<br />
Red Hat provides a utility called redhat-config-kickstart to assist with the<br />
creation of Kickstart files. When you launch this utility, you see the Kickstart<br />
Configurator window (Figure 6-5).<br />
Figure 6-5 redhat-config-kickstart main window<br />
<strong>The</strong> separate sections of this application allow you to specify different attributes<br />
of the installation.<br />
► Basic Configuration<br />
You can configure language support, keyboard, mouse, and root password<br />
here. You may also specify whether you prefer a text installation instead of a<br />
graphical installation. Lastly you may choose if the system should reboot after<br />
the automated installation is complete.<br />
► Installation Method<br />
You can choose an upgrade or clean installation here. An upgrade usually<br />
allows the preservation of existing data by upgrading the installed applications<br />
112 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
from a previous version of Red Hat. You can also specify the source of the<br />
installation (NFS, HTTP, CDROM, etc.) here.<br />
► Boot Loader Options<br />
Select the type of bootloader (GRUB or LILO) along with any relevant options<br />
from here. You may also specify kernel parameters from this window.<br />
► Partition Information<br />
You specify the partition size and type information here. You also specify the<br />
action to take with existing partitions and new disks.<br />
► Network Configuration<br />
Define population network information, such as IP, DNS, and routers, from<br />
here.<br />
► Authentication<br />
You may configure NIS, Keberos, and other authentication server options<br />
here. You may also specify local password encryption and security.<br />
► Firewall Configuration<br />
Configure iptables rules from here.<br />
► Xconfiguration<br />
Select the video card, resolution, monitor, and window manager here.<br />
► Package Selection<br />
Select which programs are installed on the system here.<br />
► Pre-Installation Script<br />
Identify the commands to run before the installation.<br />
► Post-Installation Script<br />
Identify the commands to run after the installation.<br />
When all your options are specified, save the file. This automatically checks for<br />
any missing options. If everything is correct, you may now use the ks.cfg file in an<br />
unattended installation.<br />
Tip: We found that the redhat-config-kickstart program, by default, places a<br />
skipx parameter in the ks.cfg file. We had to comment out this prior to using<br />
ks.cfg. Otherwise, the installation would fail.<br />
Chapter 6. Installing Linux 113
6.3.2 SuSE AutoYaST<br />
SLES provides a YaST applet to assist with the creation and editing of AutoYAST<br />
files. You can launch it from the Misc section of YaST, as shown in Figure 6-6.<br />
Figure 6-6 AutoYaST module within YaST<br />
114 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
After the AutoYaST editor is created, you see a window similar to the one in<br />
Figure 6-7.<br />
Figure 6-7 Main AutoYaST window<br />
You can use the following sections to configure your AutoYaST file.<br />
► Software<br />
Select the packages to install, and configure the automatic update client.<br />
► Hardware<br />
Configure partitioning, audio, printing, and X.<br />
Chapter 6. Installing Linux 115
► System<br />
Set the general system information. You also configure the language, time<br />
zone, and other locale-related settings here. And specify boot loader, logging,<br />
and runlevel information here.<br />
► Network Devices<br />
Set network adapter information. Kernel module information as well as IP<br />
details can be set here.<br />
► Network Services<br />
Network clients and daemons can be configured from here. <strong>The</strong>re are over 15<br />
different daemons to choose from. Refer to the Network Services section for<br />
more information.<br />
► Security and Users<br />
From here, you may create users that you want to be present after install. You<br />
may also change the security policy to make things such as password<br />
changes more stringent. And you can configure VPN and firewall settings<br />
from here.<br />
► Misc<br />
This section allows you to add complete configuration files, or to add special<br />
scripts to run before and after installation.<br />
Tip: AutoYaST also lets you import Kickstart files from Red Hat or Alice (a<br />
precursor to AutoYaST) files. You do this by selecting File → Import.<br />
After you are satisfied with the selections, save the file. This validates that the<br />
required minimum options are populated prior to saving.<br />
6.4 Performing an unattended installation<br />
<strong>The</strong> unattended installation process needs to pass the zImage.initrd the location<br />
of the unattended answer file during boot. Unfortunately, BOOTP does not have a<br />
facility to provide anything more than the location to the zImage.initrd and the<br />
tftpboot server to the client. To pass the required parameters, they need to be<br />
input into Open Firmware.<br />
116 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
6.4.1 Open Firmware<br />
For instructions about accessing the Open Firmware prompt, refer to “Activating<br />
the Open Firmware interfaces” on page 94.<br />
When you reach the Open Firmware prompt, pass bootparm through the boot-file<br />
variable. Example 6-8 and Example 6-9 show the settings for SuSE and Red Hat<br />
respectively.<br />
Example 6-8 Open Firmware settings for SuSE<br />
0 > setenv boot-file autoyast=nfs://192.168.1.113:/mnt/SUSE-Source/autoyast.xml<br />
install=nfs://192.168.1.113:/mnt/SLES-Source/SLES9/ netdevice=eth1<br />
Example 6-9 Open Firmware settings for Red Hat<br />
0 > setenv boot-file ks=nfs:192.168.1.112:/mnt/install-images/ks.cfg<br />
ksdevice=eth1<br />
After the installation is complete, remember to clear the boot-file value. If this<br />
value is not cleared, it is passed to the kernel as bootparm next time you boot.<br />
You can clear the value by using the nvsetenv command. We recommend that<br />
you run this command after the installation completes via either AutoYaST or<br />
Kickstart, as appropriate.<br />
This command is provided by the ppc64-utils RPM in RHEL and the util-linux<br />
RPM in SLES. <strong>The</strong> syntax to clear the boot-file variable is:<br />
# /sbin/nvsetenv boot-file ""<br />
6.4.2 mkzimage_cmdline: SuSE only<br />
mkzimage_cmdline is a utility contained in the SLES 9 lilo RPM. <strong>The</strong> SuSE kernel<br />
has special markers that allow mkzimage_cmdline to pass bootparm information<br />
directly through the zImage.initrd. Example 6-10 shows the syntax.<br />
Example 6-10 mkzimage_cmdline<br />
# /lib/lilo/chrp/mkzimage_cmdline -a 1 -c -s<br />
"autoyast=nfs://192.168.1.113:/mnt/SUSE-Source/autoyast.xmlinstall=nfs://192.16<br />
8.1.113:/mnt/SLES-Source/SLES9/ netdevice=eth1" netboot.suse.img<br />
# /lib/lilo/chrp/mkzimage_cmdline netboot.suse.img<br />
cmd_line size:512<br />
cmd_line: autoyast=nfs://192.168.1.113:/mnt/SUSE-Source/autoyast.xml<br />
install=nfs://192.168.1.113:/mnt/SLES-Source/SLES9/ netdevice=eth1<br />
active: 1<br />
Chapter 6. Installing Linux 117
<strong>The</strong> syntax for mkzimage_cmdline is straightforward. When entered with only a file<br />
name parameter. <strong>The</strong> current bootparm is displayed. You can use the following<br />
options:<br />
-h Display help.<br />
-v Display version.<br />
-a 0|1 Disable/enable command line. Overrides bootparm passed from<br />
Open Firmware.<br />
-s STRING Stores STRING as bootparm in zImage.initrd.<br />
-c Clears previously stored bootparm from zImage.initrd before<br />
applying a new one.<br />
6.5 Performing a network installation<br />
When installing several machines, we recommend that you perform a network<br />
installation instead of a CD-based installation. <strong>The</strong> speed and ability to serve<br />
multiple machines makes it ideal for installing to a full complement of<br />
<strong>BladeCenter</strong> <strong>JS20</strong>s.<br />
Attention: <strong>The</strong> cited examples are provided for illustration purposes only. For<br />
detailed instructions about how to install each distribution, refer to the product<br />
documentation.<br />
6.5.1 SUSE LINUX Enterprise Server 9<br />
For the SUSE LINUX Enterprise Server 9, follow this procedure:<br />
1. After you load the zImage.initrd file, you see a language display similar to the<br />
one in Figure 6-8. In this example, we selected English.<br />
118 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Select the language.<br />
1) Bosnia<br />
2) Cestina<br />
3) Deutsch<br />
4) English<br />
5) Español<br />
6) Français<br />
7) Hellenic<br />
8) Italiano<br />
9) Japanese<br />
10) Magyar<br />
11) Nederlands<br />
12) Polski<br />
13) Português<br />
14) Português Brasileiro<br />
15) Russian<br />
16) Slovencina<br />
><br />
Figure 6-8 Language selection<br />
2. After you select the language, the main installer display appears with a<br />
number of options. In this example, we chose to load additional kernel<br />
modules, as shown in Figure 6-9.<br />
>>> Linuxrc v1.6 (Kernel 2.6.5-7.69-pseries64) (c) 1996-2004 SUSE LINUX AG<br />
3. We select the install option and start the installation, as shown in Figure 6-10.<br />
Start Installation or System<br />
1) Start Installation or Update<br />
2) Boot Installed System<br />
3) Start Rescue System<br />
><br />
Figure 6-10 Installation menu<br />
4. We instruct the installer to use the network as a source for the installation, as<br />
shown in Figure 6-11.<br />
Choose the source medium.<br />
1) CD-ROM<br />
2) Network<br />
3) Hard Disk<br />
><br />
Figure 6-11 Source menu<br />
5. We chose NFS as the protocol to use from the menu in Figure 6-12.<br />
Choose the network protocol.<br />
1) FTP<br />
2) HTTP<br />
3) NFS<br />
4) TFTP<br />
><br />
Figure 6-12 Protocol menu<br />
120 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
6. We chose the following site-specific information. You may have different<br />
information for your environment, but the general process is the same as<br />
shown in Figure 6-13.<br />
Automatic configuration via DHCP?<br />
1) Yes<br />
2) No<br />
> 1<br />
Sending DHCP request...<br />
Enter the IP address of the NFS server [192.168.1.113]><br />
Enter the directory on the server<br />
[/mnt/SLES/SLES9-RC1/]><br />
Figure 6-13 Installation location<br />
With these steps complete the normal YaST process begins.<br />
Tip: With this, the network specific portion of the installation is complete. Refer<br />
to the product documentation for further information.<br />
6.5.2 Red Hat Enterprise Linux 3<br />
After the zImage.initrd is served by the TFTP boot server, the installation process<br />
is the same as a CD-based installation. Refer to the product documentation for<br />
more information regarding this.<br />
Chapter 6. Installing Linux 121
122 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Chapter 7. Installing AIX on the <strong>JS20</strong><br />
7<br />
This chapter discusses NIM. It explains how to configure NIM to install AIX and<br />
how to install AIX on an <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong> using NIM.<br />
<strong>BladeCenter</strong> <strong>JS20</strong> supports the installation of AIX 5.2f and later. We recommend<br />
that you install AIX using a network installation via NIM. NIM is a unified console<br />
that allows the management of several AIX logical partitions (LPARs) or<br />
machines. You may install AIX on new machines or manage existing ones from a<br />
unified interface.<br />
Tip: Prior to installing any operating system, we recommend that you upgrade<br />
all firmware to the latest level. See 5.4, “Firmware” on page 79, for instructions<br />
about how to complete this task.<br />
To use NIM, there should be an available AIX instance with NIM configured on<br />
the same network segment as the <strong>BladeCenter</strong>. In the interest of speed and<br />
security, a private dedicated subnet is recommended.<br />
Our NIM master was configured on a pSeries 690 connected to the same local<br />
area network (LAN) as the <strong>BladeCenter</strong>. This p690 was running the same level of<br />
AIX that we installed on the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
Note: An installation of AIX may be several gigabytes in size. We recommend<br />
that you use a 100 Mb or faster network for NIM-based installations.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 123
7.1 Minimal NIM installation<br />
Follow these steps to install NIM:<br />
1. Launch the Web-based Systems Manager (WebSM) and point to the server<br />
that is running NIM. Select Configure the Network Installation<br />
Management environment from the NIM object shown in Figure 7-1.<br />
Figure 7-1 NIM configuration window<br />
2. In the window that opens (Figure 7-2), select Configure this host as NIM<br />
Master.<br />
124 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Figure 7-2 Configuring as a NIM Master<br />
3. Select the network interface that is bound to the same network as the<br />
<strong>BladeCenter</strong>, and name it appropriately, as shown in Figure 7-3.<br />
Figure 7-3 Selecting the NIM interface and network name<br />
Chapter 7. Installing AIX on the <strong>JS20</strong> 125
4. In the next window (Figure 7-4), select Initialize Minimal NIM Environment.<br />
Figure 7-4 Choosing to initialize minimal NIM environment<br />
5. Click View Settings to confirm the gathered settings and complete the<br />
minimal NIM environment as shown in Figure 7-5.<br />
Figure 7-5 Finalizing the NIM environment settings<br />
126 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
7.2 Adding resources<br />
Follow these steps to add resources:<br />
1. With the minimal NIM environment complete, configure lpp_sources and a<br />
SPOT on the NIM master. Return to the NIM configuration window within<br />
WebSM and select Add a new resource to the Network Installation<br />
Management environment.<br />
2. In the Add New Resource window (Figure 7-6), select the lpp_source &<br />
SPOT option.<br />
Figure 7-6 lpp_source and SPOT configuration<br />
Chapter 7. Installing AIX on the <strong>JS20</strong> 127
3. In the Configure NIM Environment window (Figure 7-7), select the medium<br />
that you will use for your AIX source. We use the DVD RAM drive.<br />
Figure 7-7 AIX source<br />
4. As shown in Figure 7-8, select the default values and begin populating the<br />
NIM resources.<br />
Figure 7-8 Default resource settings<br />
128 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
5. Click View Settings, as shown in Figure 7-9, to confirm the settings.<br />
Assuming a CD-based installation, insert the first CD and proceed to<br />
complete the addition of the new resources.<br />
Figure 7-9 Completing the resource installation<br />
Tip: This step make take some time to complete, since many files must be<br />
incorporated into the resource.<br />
Chapter 7. Installing AIX on the <strong>JS20</strong> 129
7.3 Adding machine information<br />
With the resources added, add information regarding the machine on which you<br />
want to perform a base operating system (BOS) installation.<br />
1. From the main NIM menu (Figure 7-10), select Add a new Machine to the<br />
Network Installation Management environment.<br />
Figure 7-10 Adding new machine information to NIM<br />
130 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
2. In the Add New Machine window (Figure 7-11), enter the host name of the<br />
server that you want to add to the NIM environment from this window.<br />
Figure 7-11 Host name of new machine<br />
Tip: <strong>The</strong> NIM server must be able to resolve the host name of the new<br />
server. Ensure that this host name can be resolved by DNS or a hosts file.<br />
Chapter 7. Installing AIX on the <strong>JS20</strong> 131
3. In the next window (Figure 7-12), select Multiprocessor and confirm the<br />
other metrics of the machine that you are adding to the NIM environment.<br />
Figure 7-12 Entering machine-specific information<br />
132 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
4. As shown in Figure 7-13, enter the MAC address for the machine that you are<br />
adding to the Network adapter hardware address field and proceed to<br />
complete the addition of the new machine to the NIM environment.<br />
Figure 7-13 Entering MAC address information<br />
With the NIM resources created and the machine object created, simply proceed<br />
with the installation.<br />
NIM allows you to install new operating systems on pSeries servers by booting<br />
the machine via BOOTP. This diskless or network installation makes it simple to<br />
install new operating systems to servers in a matter of minutes.<br />
Chapter 7. Installing AIX on the <strong>JS20</strong> 133
7.4 Preparing the NIM master<br />
With all the NIM resources in place, it is simple to begin the installation of a new<br />
operating system to the target machine, in this case a <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
1. Launch WebSM and view the configured machine objects from the NIM<br />
section.<br />
2. Right-click the object for the desired machine and select Install Operating<br />
System, as shown in Figure 7-14.<br />
Figure 7-14 Installing an operating system from NIM<br />
3. Select the location of the SPOT and lpp_source resources created earlier.<br />
134 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
4. Proceed with the installation by setting the BOS preferences as shown in<br />
Figure 7-15.<br />
Figure 7-15 Setting BOS preferences<br />
Tip: To fully automate the AIX installation, create a bosinst_data resource and<br />
associate it with the operating system installation in NIM settings. Although<br />
this is beyond the scope of this book, you can find instructions in the AIX<br />
product manuals.<br />
7.5 Preparing the client<br />
With the NIM server configuration complete, simply boot the <strong>BladeCenter</strong> <strong>JS20</strong><br />
from the network as explained in 5.5.3, “Open Firmware interfaces” on page 94.<br />
When the NIM master supplies the boot image via TFTP, the AIX installation<br />
begins just as though you booted from the CD.<br />
Chapter 7. Installing AIX on the <strong>JS20</strong> 135
1. On the first panel, select the desired language. We selected English, as<br />
shown in Figure 7-16.<br />
******* Please define the System Console. *******<br />
Type a 1 and press Enter to use this terminal as the<br />
system console.<br />
Pour definir ce terminal comme console systeme, appuyez<br />
sur 1 puis sur Entree.<br />
Taste 1 und anschliessend die Eingabetaste druecken, um<br />
diese Datenstation als Systemkonsole zu verwenden.<br />
Premere il tasto 1 ed Invio per usare questo terminal<br />
come console.<br />
Escriba 1 y pulse Intro para utilizar esta terminal como<br />
consola del sistema.<br />
Escriviu 1 1 i premeu Intro per utilitzar aquest<br />
terminal com a consola del sistema.<br />
Digite um 1 e pressione Enter para utilizar este terminal<br />
como console do sistema.<br />
Figure 7-16 Installation language panel<br />
2. In the next panel (Figure 7-17), you may proceed with the default installation<br />
options or customize them. We chose to customize them.<br />
Welcome to Base Operating System<br />
Installation and Maintenance<br />
Type the number of your choice and press Enter. Choice is indicated by >>>.<br />
>>> 1 Start Install Now with Default Settings<br />
Figure 7-17 Operating system selection<br />
136 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
2 Change/Show Installation Settings and Install<br />
3 Start Maintenance Mode for System Recovery<br />
88 Help ?<br />
99 Previous Menu<br />
>>> Choice [1]:
3. In the Installation and Settings window (Figure 7-18), we proceeded with the<br />
default installation type and language environment values.<br />
Installation and Settings<br />
Either type 0 and press Enter to install with current settings, or type the<br />
number of the setting you want to change and press Enter.<br />
1 System Settings:<br />
Method of Installation.............Preservation<br />
Disk Where You Want to Install.....hdisk0<br />
2 Primary Language Environment Settings (AFTER Install):<br />
Cultural Convention................C (POSIX)<br />
Language...........................C (POSIX)<br />
Keyboard...........................C (POSIX)<br />
3 More Options (Desktop, Security, Kernel, Software, ...)<br />
>>> 0 Install with the settings listed above.<br />
+-----------------------------------------------------<br />
88 Help ? | WARNING: Base Operating System Installation will<br />
99 Previous Menu |destroy or impair recovery of SOME data on the<br />
|destination disk hdisk0.<br />
>>> Choice [0]:<br />
Figure 7-18 Installation and language options<br />
Chapter 7. Installing AIX on the <strong>JS20</strong> 137
4. As illustrated in Figure 7-19, we elected to include additional options such as<br />
the 64-bit kernel and JFS2. We then selected 0 and began the installation<br />
process.<br />
Install Options<br />
1. Enable Trusted Computing Base.................................... no<br />
2. Enable CAPP and EAL4+ Technology................................. no<br />
(English only, 64-bit kernel enablement, JFS2 file systems)<br />
3. Enable 64-bit Kernel............................................. yes<br />
4. Create JFS2 File Systems......................................... yes<br />
5. Graphics Software................................................ yes<br />
6. Documentation Services Software.................................. no<br />
7. Enable System Backups to install any system...................... no<br />
(Installs all devices and kernels)<br />
>>> 8. Install More Software<br />
Figure 7-19 Kernel options<br />
138 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
0 Install with the settings listed above.<br />
88 Help ?<br />
99 Previous Menu<br />
>>> Choice [8]:<br />
Tip: As with any operating system, AIX enables you to customize many<br />
different options during the installation process. We have provided a sample of<br />
what you can select.
Chapter 8. System management<br />
scenarios<br />
8<br />
Your installation may be comprised of a single <strong>BladeCenter</strong> chassis with only a<br />
few <strong>BladeCenter</strong> <strong>JS20</strong>s. If so, you may find that the capabilities provided by the<br />
<strong>BladeCenter</strong> Management Module Web interface and command line interface<br />
(CLI) are sufficient to meet your system management requirements. For larger<br />
installations comprised of multiple <strong>BladeCenter</strong> chassis and many <strong>BladeCenter</strong><br />
<strong>JS20</strong>s, consider using <strong>IBM</strong> Director, <strong>IBM</strong> Cluster Systems Management (CSM),<br />
or both.<br />
We demonstrate how to use many of the functions provided by the <strong>BladeCenter</strong><br />
Management Module Web interface and CLI in Chapter 5, “Hardware setup” on<br />
page 69.<br />
This chapter provides information about setting up and configuring the <strong>IBM</strong><br />
Director and the <strong>IBM</strong> Cluster Systems Management facilities. It also describes<br />
several systems management scenarios for the <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 139
8.1 <strong>IBM</strong> Director<br />
<strong>IBM</strong> Director is a comprehensive systems-management solution. A powerful<br />
suite of tools and utilities, <strong>IBM</strong> Director automates many of the processes<br />
required to manage systems proactively, including preventive maintenance,<br />
diagnostic monitoring, troubleshooting, and more. It offers a graphical user<br />
interface (GUI) that provides system administrators easy access to both local<br />
and remote systems. <strong>IBM</strong> Director can be used in environments with multiple<br />
operating systems (heterogeneous environments).<br />
At the time of writing, the current version of <strong>IBM</strong> Director was Version 4.2.<br />
<strong>IBM</strong> Director is the industry-leading client/server workgroup manager. <strong>IBM</strong><br />
Director’s tools provide customers with flexible capabilities to realize maximum<br />
system availability and lower IT costs. With <strong>IBM</strong> Director, IT administrators can<br />
view and track the hardware configuration of remote systems in detail and<br />
monitor the usage and performance of critical components, such as processors,<br />
disks, and memory.<br />
Extensions to <strong>IBM</strong> Director are available for clients who want advanced<br />
capabilities. <strong>The</strong>se extensions include:<br />
► Server Plus Pack<br />
► Software Distribution Premium Edition<br />
► Remote Deployment Manager<br />
► Application Workload Manager<br />
Some of the features that you find in <strong>IBM</strong> Director include:<br />
► Self-managing, smart tools that are automated, and proactive capabilities that<br />
reduce IT costs and maximize uptime<br />
► Support for non-<strong>IBM</strong> hardware<br />
Innovative use of industry standards from Common Information Model (CIM)<br />
to Simple Network Mail Protocol (SNMP) enables heterogeneous hardware<br />
management, protecting your existing IT investment.<br />
► Seamless Integration<br />
<strong>IBM</strong> Director protects your investments in other management packages,<br />
complementing them with more extensive hardware manageability.<br />
► Single-click management GUI<br />
A convenient user interface delivers the ability to drag and drop tasks to<br />
specific systems or groups of systems.<br />
140 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
► Integrated, centralized SQL database<br />
An internal database makes system-related data available, even when the<br />
specific system is not directly available.<br />
► Multiple operating system support<br />
<strong>IBM</strong> Director smoothly handles a variety of popular operating systems.<br />
► Comprehensive <strong>BladeCenter</strong> support<br />
<strong>IBM</strong> Director provides an easy, single point of deployment and management<br />
of new blade server architectures.<br />
► Consistent framework that can be extended with plug-ins for advanced<br />
management<br />
► CLI support for many <strong>IBM</strong> Director management tasks<br />
<strong>IBM</strong> Director 4.20 has many new features, support for new operating systems,<br />
and enhancements to existing functions. Together, these changes provide<br />
significantly richer and broader capabilities for superior hardware management.<br />
Key enhancements include:<br />
► Expanded Linux support including aggregated, color-coded systems health<br />
status for systems running Linux<br />
► Convenient subscription service included with the <strong>IBM</strong> Director license to<br />
proactively deliver the latest product updates<br />
► Enhanced optional Server Plus Pack, a collection of advanced tools to help<br />
optimize server performance and availability<br />
Some of the new features include XML report format for the Capacity<br />
Manager and System Availability tools, the ability to schedule System<br />
Availability reports, and Active PCI Manager support for Linux.<br />
► <strong>The</strong> latest hardware support including support for additional hardware<br />
industry standards for manageability, ASF 2.0 and IPMI, to further facilitate<br />
the management of heterogeneous hardware environments<br />
► A multitude of new features and refinements to make <strong>IBM</strong> Director even<br />
easier to install and use<br />
Chapter 8. System management scenarios 141
8.1.1 Setting up an <strong>IBM</strong> Director Server<br />
Before Director clients can be installed, you must first configure the Director<br />
server. <strong>The</strong> following instructions explain how we set up our server.<br />
1. Mount the Director CD.<br />
sysmgr:/ # mount -t iso9660 -o map=off /dev/cdrom /media/cdrom<br />
Important: It is critical that the CD is mounted in this manner. If you do not<br />
specify -o map=off, the capitalization of RPM names is not preserved and<br />
the installation script fails.<br />
2. With the CD mounted, copy the contents of the appropriate directory to the<br />
hard drive of the server.<br />
sysmgr:/root/Director-Source/ # cp<br />
/media/cdrom/director/server/linux/i386/* .<br />
3. With the sources copied to the hard drive, launch the installer.<br />
sysmgr:/root/Director-Source/ # ./dirinstall<br />
4. After the Director code is installed, configure a database for it to use. Because<br />
PostgreSQL ships as a standard part of SLES, we chose to use it.<br />
Example 8-1 shows how we configured it.<br />
Example 8-1 Configuring PostgreSQL<br />
sysmgr:/root/Director-Source/ # cd /usr/share/pgsql<br />
sysmgr:/usr/share/pgsql/ # ln -s jdbc7.1-1.2.jar postgresql.jar<br />
sysmgr:/usr/share/pgsql/ # cd /etc/TWGserver/<br />
sysmgr:/etc/TWGServer/ # echo export CLASSPATH=/usr/share/pgsql/psotgresql.jar<br />
> setup_env<br />
sysmgr:/etc/TWGServer/ # chmod u+x setup_env<br />
Important: We assume that PostgreSQL is configured and running<br />
properly on the system. Refer to the documentation regarding how to do<br />
this.<br />
5. By default, PostgreSQL on SLES does not bind to a TCP/IP socket. To enable<br />
this, modify /etc/sysconfig/postgresql and add the following line:<br />
POSTGRES_OPTIONS="-i"<br />
6. Begin the database configuration, as shown in Example 8-2.<br />
Example 8-2 Launching the database configuration<br />
sysmgr:/etc/TWGServer/ # passwd postgres<br />
Changing password for postgres<br />
142 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
New password:<br />
Re-enter new password:<br />
Password changed<br />
sysmgr:/etc/TWGServer/ # cd /opt/<strong>IBM</strong>/director/bin/<br />
sysmgr:/opt/<strong>IBM</strong>/director/bin # ./cfgdb<br />
7. <strong>The</strong> installer launches. In the first window (Figure 8-1), select PostgreSQL.<br />
Figure 8-1 Selecting the type of database<br />
Chapter 8. System management scenarios 143
8. Enter the database connection information, as shown in Figure 8-2. Click<br />
Next to complete the configuration of the database.<br />
Figure 8-2 Database connection information<br />
144 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
9. We decided to use encrypted communication for our <strong>IBM</strong> Director<br />
environment. To do this, execute cfgsecurity as shown in Figure 8-3.<br />
sysmgr:/opt/<strong>IBM</strong>/director/bin # cfgsecurity<br />
Do you want <strong>IBM</strong> Director to encrypt data communications on this system?<br />
0 - No<br />
1 - Yes<br />
Enter number corresponding to desired security setting (0 is default):<br />
1<br />
Which encryption algorithm do you want to use?<br />
1 - DES encryption (56-bit key)<br />
2 - Triple DES encryption (128-bit key)<br />
2<br />
0 - No encryption<br />
Encryption set or unset successfully.<br />
All <strong>IBM</strong> Director servers and encrypted systems are set to "secured"<br />
automatically.<br />
Setting secured or unsecured system successful.<br />
Figure 8-3 Setting Director server encryption<br />
10.After you configure the database and security, start the Director daemon.<br />
sysmgr:/opt/<strong>IBM</strong>/director/bin # ./twgstart<br />
With the daemon started, the Director server configuration is now complete.<br />
8.1.2 Installing an <strong>IBM</strong> Director Agent<br />
Follow these steps to install an <strong>IBM</strong> Director Agent.<br />
1. Mount the Director CD and copy the files as explained in steps 1 and 2 on<br />
page 142. This time, substitute /media/cdrom/director/server/linux/i386/ with<br />
/media/cdrom/director/agent/linux/ppc/ in step 2.<br />
Chapter 8. System management scenarios 145
2. With the sources copied to the hard drive, launch the installer, as shown in<br />
Figure 8-4.<br />
blade:/root/Director-Source/ # ./dirinstall<br />
Attempting to install ITDAgent-4.12-1.ppc.rpm<br />
Preparing...<br />
#################################################<br />
ITDAgent<br />
#################################################<br />
To start the <strong>IBM</strong> Director Agent manually, run /opt/ibm/director/bin/twgstart<br />
Installation of selected components is successful.<br />
To start the <strong>IBM</strong> Director Agent manually, run /opt/ibm/director/bin/twgstart<br />
Figure 8-4 Installing the Director Agent<br />
3. After the installation is complete, configure security as desired. We chose to<br />
use encryption, as shown in Figure 8-5.<br />
blade:/etc/TWGServer/ # cd /opt/<strong>IBM</strong>/director/bin/<br />
blade:/opt/<strong>IBM</strong>/director/bin/ # cfgsecurity<br />
Do you want <strong>IBM</strong> Director to encrypt data communications on this system?<br />
0 - No<br />
1 - Yes<br />
Enter number corresponding to desired security setting (0 is default):<br />
1<br />
Encryption set or unset successfully.<br />
All <strong>IBM</strong> Director servers and encrypted systems are set to "secured"<br />
automatically.<br />
Setting secured or unsecured system successful.<br />
Figure 8-5 Configuring Director Agent security<br />
4. With the security set, start the Director Agent.<br />
blade:/opt/<strong>IBM</strong>/director/bin # ./twgstart<br />
146 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
8.1.3 Using Director to manage <strong>JS20</strong>s<br />
With the agent installed, you can add the <strong>BladeCenter</strong> <strong>JS20</strong> to the Director<br />
console as another managed server. For details about how to do this, see the<br />
<strong>IBM</strong> Director 4.12 Installation and Configuration Guide, SC09-P290-50.<br />
<strong>The</strong> following subset of Director management features are available for the<br />
<strong>BladeCenter</strong> <strong>JS20</strong>:<br />
► Software Distribution Standard Edition<br />
► Software Distribution Premium Edition<br />
► Update Assistant<br />
► Inventory Hardware<br />
► Inventory Software<br />
► Event Action PLans<br />
► Hardware Health Status<br />
► Resource Monitors<br />
► Eventlog<br />
► Agent Discovery<br />
► Process Management<br />
► SNMP Alerts<br />
► SNMP Browser<br />
► Rack Manager<br />
► Remote Session<br />
► File Transfer<br />
8.2 <strong>IBM</strong> Cluster Systems Management<br />
<strong>BladeCenter</strong> <strong>JS20</strong> installations may involve clusters of many blade servers. Many<br />
of these installations use <strong>IBM</strong> Cluster Systems Management to both install and<br />
manage <strong>BladeCenter</strong> <strong>JS20</strong>s.<br />
CSM for Linux has been under development since early 2000 and was first<br />
released as a product in June 2001. <strong>The</strong> original requirement for this product was<br />
to provide common cluster management for Linux and AIX, leveraging <strong>IBM</strong>’s<br />
existing cluster software portfolio to work with Open Source packages and best<br />
practices from the Linux cluster market.<br />
CSM provides basic cluster management functions, such as hardware control<br />
and operating system and software installation. It also provides significant<br />
value-added functions such as configuration file management, a robust<br />
monitoring infrastructure, automated event management, diagnostic probes, and<br />
common management of AIX and Linux clusters.<br />
Chapter 8. System management scenarios 147
As CSM was being developed, the Extreme Cluster Administration Toolkit<br />
(xCAT), a script-based package, was developed by <strong>IBM</strong>’s advanced technical<br />
sales support team. It was provided to clients who purchase Linux clusters based<br />
on <strong>IBM</strong> servers. It was also used by <strong>IBM</strong> Global Services to address their need<br />
for tools to deploy and manage Linux clusters. In the interim, xCAT became a<br />
proving ground for new concepts and practices centered on the management of<br />
Linux clusters. It can be a model for collaborative development between <strong>IBM</strong> and<br />
our clients.<br />
At the time of the development of this book, CSM was available in its fourth<br />
release (V1.4.0.4) with expanded scalability and hardware support. Many xCAT<br />
functions have been integrated into CSM, along with utilities to automate the<br />
process of migrating from xCAT to CSM. Through the use of CSM, clients who<br />
have implemented xCAT can now have many of these functions with full product<br />
support from <strong>IBM</strong>.<br />
<strong>The</strong> key benefits of CSM can help to reduce costs, provide higher availability of<br />
the cluster for productive use, and improve system utilization. Customers who<br />
have existing AIX 5L operating system-based cluster systems can leverage those<br />
skills to manage their Linux clusters.<br />
System administrators can automate repetitive installation and configuration<br />
tasks and automate problem determination and recovery. <strong>The</strong>y can monitor and<br />
report health information and resource utilization, and automatically recover from<br />
node, storage, or network failures. This can lead to overall simplification of cluster<br />
administration.<br />
This section describes a fast-path CSM implementation to install and manage<br />
<strong>BladeCenter</strong> <strong>JS20</strong>s. For a more comprehensive view, refer to the CSM for Linux<br />
Planning and Installation Guide, SA22-7853.<br />
Our implementation of CSM assumes the type of network configuration<br />
described in 4.1.1, “Minimal network requirements” on page 54. Review this<br />
along with other CSM planning considerations described in 4.3.2, “<strong>IBM</strong> Cluster<br />
Systems Management” on page 64.<br />
8.2.1 Setting up a CSM management node<br />
<strong>The</strong> first step in installing CSM is to set up a management node. We focus on<br />
how to set up a management node based on an xSeries server. We believe this<br />
is a common scenario for users of <strong>BladeCenter</strong> <strong>JS20</strong>s, particularly if they also<br />
have other blade servers that use Intel central processing units (CPUs).<br />
Using an xSeries server to install and manage <strong>BladeCenter</strong> <strong>JS20</strong>s cluster nodes<br />
introduces some complications that are not required when the management<br />
148 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
node is the same architecture as the compute nodes. We focus our discussion<br />
around these issues.<br />
We use CSM to both install and manage <strong>BladeCenter</strong> <strong>JS20</strong>s. <strong>The</strong>refore, we must<br />
use a compatible operating system on both the management node and the<br />
<strong>BladeCenter</strong> <strong>JS20</strong> cluster nodes. <strong>The</strong> steps we require to set up the CSM<br />
management node are:<br />
1. Install a supported operating system.<br />
2. Configure network interfaces and set up host name resolution.<br />
3. Obtain the latest CSM software and prerequisites.<br />
4. Install the CSM base RPM.<br />
5. Run the CSM installation program.<br />
6. Copy the operating system, service packs, CSM code, and prerequisites for<br />
<strong>BladeCenter</strong> <strong>JS20</strong> compute nodes.<br />
7. Accept the CSM license.<br />
8. Set up hardware control point passwords.<br />
9. Verify the installation.<br />
We review each of these steps in the sections that follow. For our tests, we<br />
performed a standard installation of SUSE LINUX Enterprise Server (SLES) 9 for<br />
x86 on our management node directly from the distribution CDs.<br />
<strong>The</strong> management node network configuration<br />
Our implementation of CSM assumes the type of network configuration<br />
described in 4.1.1, “Minimal network requirements” on page 54.<br />
<strong>The</strong> CSM management node is connected to both the hardware management<br />
subnet and the operating system management subnet using separate network<br />
interfaces. You can use additional network interfaces to connect the management<br />
node to other networks if required.<br />
In our scenario, we used the IP address ranges 9.3.5/24 for the hardware<br />
management subnet and 192.168.1/24 for the operating system management<br />
subnet. <strong>The</strong>re should be no other active DHCP servers on the operating system<br />
management subnet, because the CSM management node performs the DHCP<br />
server function on that subnet.<br />
Define host names and allocate IP addresses for the following interfaces before<br />
you attempt to use CSM:<br />
► <strong>The</strong> management node network interface that connects to the operating<br />
system management subnet<br />
Chapter 8. System management scenarios 149
This is the interface that will be used to install the operating system on the<br />
<strong>BladeCenter</strong> <strong>JS20</strong> cluster nodes.<br />
► <strong>The</strong> network interface on each <strong>BladeCenter</strong> <strong>JS20</strong> that connects to the<br />
operating system management subnet<br />
► <strong>The</strong> external network interface of each <strong>BladeCenter</strong> Management Module<br />
You can either set up a hosts file on the management node or use a Domain<br />
Name System (DNS) server to define the host name to IP address mapping. In<br />
our scenario, we used a hosts file as shown in Example 8-3.<br />
Example 8-3 /etc/hosts on management node<br />
127.0.0.1 localhost<br />
192.168.1.20 mgr<br />
192.168.1.117 blade1<br />
192.168.1.118 blade2<br />
192.168.1.119 blade3<br />
9.3.5.190 bcmm<br />
Obtaining the latest CSM software and prerequisites<br />
You can obtain the latest CSM software from:<br />
http://techsupport.services.ibm.com/server/cluster/fixes/csmfixhome.html<br />
Check this Web site for the latest CSM software even if you have CSM available<br />
on CD, because new CSM software releases occur every few months.<br />
<strong>The</strong> CSM software available at the previous Web site is under a 60-day Try and<br />
Buy license. If you are performing a permanent CSM installation, you need to<br />
have purchased an appropriate quantity of CSM licenses. You also need the full<br />
license file that is contained on the CSM product CD.<br />
We downloaded two different versions of CSM: one for the management node<br />
(xSeries based) and one for the <strong>BladeCenter</strong> <strong>JS20</strong> cluster nodes. <strong>The</strong> two<br />
versions of CSM that we downloaded were:<br />
► CSM for Linux on xSeries V1.4.0.4<br />
► CSM for Linux on pSeries V1.4.0.4<br />
Follow these steps:<br />
1. Type the following command:<br />
mkdir /tmp/csm_i386<br />
2. Now type this command:<br />
mkdir /tmp/csm_js20<br />
150 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
3. Download the two CSM tarballs into their respective directories created in<br />
steps 1 and 2.<br />
4. Change to /tmp/csm_i386 and run the following command:<br />
tar -xvzf csm-linux-1.4.0.0.i386.tar.gz<br />
5. Change to /tmp/csm_js20 and run the following command:<br />
tar -xvzf csm-linux-1.4.0.ppc64.tar.gz<br />
6. We created a directory for such prerequisite packages as Autoupdate called<br />
/tmp/csmpreq. Autoupdate is a package that is used by CSM. We<br />
downloaded Version 5.3.11-1 from the following URL and placed it into the<br />
/tmp/csmpreq directory.<br />
http://ftp.mat.univie.ac.at/pub/teschl/autoupdate/index.html<br />
Installing core CSM RPM<br />
You must be logged into the management node as root to perform this step.<br />
We installed the core CSM RPM on the management node using this command:<br />
# rpm -ivh /tmp/csm_i386/csm.core*.i386.rpm<br />
When you run the rpm command and install the CSM core, CSM sets the $PATH<br />
and $MANPATH environment variables. <strong>The</strong>refore, you must log out and then log<br />
in again to the management node after performing the installation to accept the<br />
changes to the environment variables that are required when performing<br />
subsequent installation steps.<br />
Running CSM installation program<br />
After logging back into the management node as root, we ran the CSM<br />
installation program using the following command:<br />
# installms -p /tmp/csm_i386:/tmp/csmpreq<br />
Note: <strong>The</strong> installms command is found in the /opt/csm/bin directory after you<br />
install the CMS core.<br />
<strong>The</strong> -p option of the installms command allows you to specify the location where<br />
the CSM installation program should search for the files it needs. We specified<br />
the two directories where we placed the CSM software and prerequisites for the<br />
management node hardware architecture that we were using.<br />
<strong>The</strong> installms command prompts you to insert operating system CDs if it cannot<br />
find the operating system packages that it needs in one of the directories listed<br />
after the -p option. You are also prompted to replace the Trivial File Transfer<br />
Protocol (TFTP) daemon provided with the Linux distribution with a replacement<br />
Chapter 8. System management scenarios 151
version packaged with CSM. And you are prompted to accept the licenses<br />
associated with some software packages distributed with CSM.<br />
Make sure you resolve any errors reported by the installms command before<br />
you continue with the next step.<br />
Copying the code required for <strong>BladeCenter</strong> <strong>JS20</strong>s<br />
<strong>The</strong> <strong>BladeCenter</strong> <strong>JS20</strong> has a different hardware architecture than the xSeries<br />
server that we used for our management node. <strong>The</strong>refore, you must copy the<br />
code required for the <strong>BladeCenter</strong> <strong>JS20</strong>s to the management node. This<br />
includes:<br />
► SUSE LINUX Enterprise Server 9 for <strong>IBM</strong> POWER<br />
► CSM for Linux on pSeries<br />
► CSM prerequisites<br />
We used the CSM copycds and copycsmpkgs commands to copy the operating<br />
system, service pack, CSM software, and prerequisites for the <strong>BladeCenter</strong> <strong>JS20</strong><br />
cluster nodes to the management node. Both of these commands require you to<br />
specify any attributes that are different to the operating system environment on<br />
the management node. In our scenario, we specified the InstallPkgArchitecture<br />
attribute, because the blade server cluster nodes have a different architecture<br />
than the management node.<br />
We copied the operating system CDs using the following command:<br />
# copycds InstallPkgArchitecture=ppc64<br />
We copied the CSM software and prerequisites for the <strong>BladeCenter</strong> <strong>JS20</strong> cluster<br />
nodes using this command:<br />
# copycsmpkgs -p /tmp/csm_js20:/tmp/csmpreq InstallPkgArchitecture=ppc64<br />
<strong>The</strong> copycsmpkgs command prompts you to insert operating system CDs if it<br />
cannot find the operating system packages that it needs in one of the directories<br />
listed after the -p option.<br />
Accepting the CSM licence<br />
Accept the CSM licence. If you want to use a 60-day trial licence, use the<br />
following command:<br />
# csmconfig -L<br />
If you want to accept the permanent license, mount the CSM product CD and run<br />
the following command:<br />
# csmconfig -L /media/cdrom/csmlum.full<br />
152 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Setting up hardware control point passwords<br />
CSM requires user IDs and passwords to communicate with <strong>BladeCenter</strong><br />
Management Modules so that it can:<br />
► Control power to <strong>BladeCenter</strong> <strong>JS20</strong>s<br />
► Provide remote console access to <strong>BladeCenter</strong> <strong>JS20</strong>s<br />
► Retrieve the MAC addresses for the Gigabit Ethernet interfaces on<br />
<strong>BladeCenter</strong> <strong>JS20</strong>s<br />
CSM lets you define one user ID and password for power control and a different<br />
user ID and password for remote console access for each <strong>BladeCenter</strong><br />
Management Module. Both of these profiles must match login profiles that are<br />
defined in the corresponding <strong>BladeCenter</strong> Management Module. You can<br />
configure these profiles through the <strong>BladeCenter</strong> Management Module Web<br />
interface.<br />
In a production environment, you should define two different login profiles, one<br />
for power control and the other for remote console access. You should also limit<br />
the access for each of these profiles to the minimum needed. In our test<br />
environment, we used the same profile for both power control and remote<br />
console access.<br />
We recommend that you use the same user ID and password for power control<br />
for all <strong>BladeCenter</strong> Management Modules in a cluster and define these as the<br />
default user ID and password for power control in CSM.<br />
You define a default <strong>BladeCenter</strong> Management Module user ID and password for<br />
power control using the following command:<br />
# systemid -p blade USERID<br />
<strong>The</strong> systemid command prompts you to enter the password associated with the<br />
default user ID you specified.<br />
Similarly, we recommend that you use the same user ID and password for remote<br />
console access for all <strong>BladeCenter</strong> Management Modules in a cluster and define<br />
these as the default user ID and password for remote console access in CSM.<br />
You define a default <strong>BladeCenter</strong> Management Module user ID and password for<br />
remote console access using the following command:<br />
# systemid -c -p blade USERID<br />
<strong>The</strong> systemid command prompts you to enter the password associated with the<br />
default user ID you specified. This step is necessary for the Serial over LAN<br />
(SoL) access.<br />
Chapter 8. System management scenarios 153
You can verify what user IDs you configured as the default for <strong>BladeCenter</strong> power<br />
control and remote console access by issuing the systemid command with no<br />
parameters. This displays output similar to what is shown in Example 8-4.<br />
Example 8-4 Output from systemid command<br />
blade_#DEFAULT# USERID<br />
blade_#DEFAULT#_console USERID<br />
Verifying the installation<br />
CSM is provided with an installation verification probe that can pick up many<br />
installation errors. You can run this probe by using the following command:<br />
# probemgr -p ibm.csm.ms -l 0<br />
Review the output of this command and investigate any error messages.<br />
8.2.2 Installing and managing <strong>BladeCenter</strong> <strong>JS20</strong>s using CSM<br />
After you establish a CSM management node, you can install and manage<br />
<strong>BladeCenter</strong> <strong>JS20</strong>s.<br />
<strong>The</strong> required steps are:<br />
1. Define your <strong>BladeCenter</strong> <strong>JS20</strong> cluster nodes to CSM.<br />
2. Set up the installation environment.<br />
3. Start installation and wait for completion.<br />
We examine each of these steps in more detail in the sections that follow.<br />
Defining the <strong>BladeCenter</strong> <strong>JS20</strong> cluster nodes<br />
We find that the easiest way to define <strong>BladeCenter</strong> <strong>JS20</strong> cluster nodes to CSM is<br />
to use a node definition file. <strong>The</strong>re are many other ways to define nodes in CSM.<br />
Review the CSM for Linux Planning and Installation Guide, SA22-7853, to<br />
determine the most appropriate method for your environment.<br />
<strong>The</strong> node definition file that we used for our small cluster of <strong>BladeCenter</strong> <strong>JS20</strong>s is<br />
shown in Example 8-5.<br />
Example 8-5 CSM node definition file<br />
default:<br />
ManagementServer=mgr<br />
PowerMethod=blade<br />
HWControlPoint=bcmm<br />
ConsoleMethod=blade<br />
Consol<strong>eServer</strong>Name=bcmm<br />
154 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
lade1:<br />
blade2:<br />
blade3:<br />
InstallDiskType=ide<br />
InstallPkgArchitecture=ppc64<br />
InstallAdapterName=eth1<br />
HWControlNodeId=blade1<br />
ConsolePortNum=1<br />
HWControlNodeId=blade2<br />
ConsolePortNum=2<br />
HWControlNodeId=blade3<br />
ConsolePortNum=3<br />
<strong>The</strong> default stanza of the node definition file contains the node attributes that are<br />
common to all the <strong>BladeCenter</strong> <strong>JS20</strong> cluster nodes in our cluster. You can have<br />
multiple default stanzas in a node definition file, with the new defaults applying to<br />
those stanzas listed in the node definition file after the new default stanza.<br />
<strong>The</strong> remaining stanzas define each <strong>BladeCenter</strong> <strong>JS20</strong> cluster node, and contain<br />
only those node attributes that are unique to that cluster node. <strong>The</strong> name of the<br />
stanza for each <strong>BladeCenter</strong> <strong>JS20</strong> cluster node should match the host name that<br />
you assigned to that cluster node in your hosts file or DNS.<br />
Use the following guide when you specify attribute values:<br />
► ManagementServer: Host name of management node network interface<br />
connected to the operating system management subnet<br />
► HWControlPoint: Host name of the <strong>BladeCenter</strong> Management Module<br />
► Consol<strong>eServer</strong>Name: Host name of the <strong>BladeCenter</strong> Management Module<br />
► InstallAdapterName: Set to eth1 if you are using the network configuration<br />
described in 4.1.1, “Minimal network requirements” on page 54<br />
This network configuration performs operating system network installation via<br />
eth1 to prevent issues with the SoL remote text console function, which uses<br />
eth0.<br />
► HWControlNodeId: Same name you assigned the <strong>BladeCenter</strong> <strong>JS20</strong> cluster<br />
node when you set up the hardware<br />
We recommend that you make it the same as the host name. We demonstrate<br />
how to configure this in 5.3, “Blade server configuration” on page 76.<br />
► ConsolePortNum: Bay in the <strong>BladeCenter</strong> chassis where the <strong>BladeCenter</strong><br />
<strong>JS20</strong> cluster node is installed<br />
Chapter 8. System management scenarios 155
► PowerMethod: Should always be blade for <strong>BladeCenter</strong> <strong>JS20</strong> cluster nodes<br />
► ConsoleMethod: Should always be blade for <strong>BladeCenter</strong> <strong>JS20</strong> cluster<br />
nodes<br />
After you create a node definition file for your cluster, you can define the nodes to<br />
CSM using the following command:<br />
# definenode -f /tmp/nodedefs<br />
After the definenode command runs, the management server is set up with all<br />
the node information for CSM, and you are ready to verify the node definitions.<br />
Since the actual node installation has not happened yet, you can make changes<br />
to any of the node definitions at this time. To determine whether the nodes have<br />
been defined, enter the lsnode command from the management server.<br />
After you run the lsnode command, the system responds with a line for each<br />
node that was successfully defined. If a node is not defined, it does not appear in<br />
the output for the lsnode command.<br />
To display all the information about each node, use the lsnode command from<br />
the management server with the –l (lowercase L) option, such as:<br />
# lsnode -l<br />
<strong>The</strong> system responds with a list (output) that contains the extended information<br />
for each node that was successfully defined. If a node is not defined, it does not<br />
appear in the output for the lsnode command.<br />
Some of the attributes for a node may have null values at this point. If a node is<br />
not defined, it is not installed. If you must correct the information, you can change<br />
the attributes of the nodes either by running the chnode command, or by<br />
rerunning the definenode command with the -m (modify) flag. <strong>The</strong> definenode -m<br />
command accepts a new nodedef file or a new command line and only modifies<br />
nodes that have changed attributes. To change attribute values of a node, enter<br />
the chnode command from the management server:<br />
chnode hostname attr=value<br />
Setting up the installation environment<br />
After you successfully define the node, configure how the node is to be installed<br />
in the cluster. For Red Hat AS or Red Hat EL 3 installations, use Kickstart, and<br />
run the csmsetupks command. For SUSE LINUX Enterprise Server installations,<br />
use AutoYaST, and run the csmsetupyast command. <strong>The</strong> csmsetupks and<br />
csmsetupyast commands configure the install process, collect MAC addresses,<br />
and configure the network boot of the nodes.<br />
156 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
In our lab environment using SuSE, the csmsetupyast did several things,<br />
including:<br />
► Retrieved the MAC addresses for the Gigabit Ethernet interfaces of each<br />
<strong>BladeCenter</strong> <strong>JS20</strong> cluster node<br />
► Built a Autoyast configuration file for each <strong>BladeCenter</strong> <strong>JS20</strong> cluster node<br />
from a template<br />
► Built the DHCP server configuration file /etc/dhcpd.conf<br />
Before you attempt to use csmsetupyast for the first time, we recommend that<br />
you verify that power control and remote console functionality is working<br />
correctly. You can verify that power control is working using the following<br />
command:<br />
# rpower -a query<br />
This command displays the current power status of all nodes in the cluster. If this<br />
is not successful, determine why before you proceed any further.<br />
You can verify that the remote consoles are working by using the following<br />
command:<br />
# rconsole -a<br />
This command opens a separate console window for every cluster node that is<br />
defined to CSM. If a <strong>BladeCenter</strong> <strong>JS20</strong> cluster node is not powered on, you see<br />
messages indicating that the SoL is not ready. Close each remote console<br />
window using the key sequence Ctrl+E c . before you continue further.<br />
When you are satisfied that power control and remote console functionality is<br />
working correctly, verify that the DHCP server on the management node is<br />
configured to listen on the correct network interface.<br />
Under SUSE LINUX Enterprise Server, the interfaces that the DHCP server uses<br />
to listen are configured in the file /etc/sysconfig/dhcpd. Set the<br />
DHCP_INTERFACE variable in this file to include the Ethernet interface on the<br />
management node that is connected to the operating system management<br />
subnet described in “Operating system management subnet” on page 57.<br />
Now set up the installation environment using the CSM command:<br />
# csmsetupyast -x -P<br />
We use the -x option because we already copied the necessary files to support<br />
the network installation of the operating system to the management node. If you<br />
do not specify this option, you are prompted to insert operating system CDs. We<br />
use the -P option to specify all nodes that are in the premanaged state and are<br />
waiting to be installed.<br />
Chapter 8. System management scenarios 157
<strong>The</strong> csmsetupyast command uses an XML template file found in /opt/csm/install<br />
as a default template for the Autoyast configuration files that it creates for each<br />
<strong>BladeCenter</strong> <strong>JS20</strong> cluster node. Depending upon the release of SuSE that you<br />
are using at the time, the file has the format:<br />
yastcfg.InstallDistributionNameInstallDistributionVersion-Arch.xml<br />
For example, in our lab environment, the file was called<br />
yastcfg.SLES9-ppc64.xml.<br />
If you want to use a different template, first create the template and then specify it<br />
on the csmsetupyast command using the -k option. We recommend that you<br />
copy the default template when creating new templates to ensure that you<br />
include all the elements that CSM requires in the template.<br />
Review the output of the csmsetupyast command carefully and resolve any errors<br />
before you proceed further.<br />
Starting the cluster node installation<br />
Start the cluster node installation using the installnode command. We<br />
recommend that you try to install a few nodes at first and make sure it works<br />
before you try to install many the nodes concurrently. To start installation of a<br />
single node, use the command:<br />
installnode -n blade1<br />
<strong>The</strong> installnode command takes a few minutes to complete when used with<br />
<strong>BladeCenter</strong> <strong>JS20</strong> cluster nodes. This is because the installnode command<br />
must wait until the <strong>BladeCenter</strong> <strong>JS20</strong> boot process has reached the point where<br />
it can pass network installation parameters to the firmware.<br />
When the installation is underway, the installnode command exits and displays<br />
the status for each node it was asked to install. You can monitor the status of the<br />
install as it progresses using the monitorinstall command. You can also open a<br />
remote console to watch the installation progress using the rconsole command.<br />
For more information about installing CSM, see CSM for Linux: Planning and<br />
Installation Guide, SA22-7853.<br />
158 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Abbreviations and acronyms<br />
ac alternating current<br />
AIX Advanced Interactive<br />
eXecutive<br />
BIU Bus Interface Unit<br />
BOOTP Bootstrap Protocol<br />
BOS base operating system<br />
BPU branch processing unit<br />
BSMP Blade System Management<br />
Processor<br />
CISC complex instruction set<br />
computer<br />
CLI command line interface<br />
CRLF carriage return/line feed<br />
CSM Cluster Systems<br />
Management<br />
CTR count register<br />
DC direct current<br />
DDR double data rate<br />
DHCP Dynamic Host Configuration<br />
Protocol<br />
DNS Domain Name Server<br />
ECC Error Correction Code<br />
ERAT effective to real address<br />
translation<br />
ESM Ethernet Switch Module<br />
FPR Floating Point Register<br />
FPU Floating Point Unit<br />
FSB Front Side Bus<br />
FTP File Transfer Protocol<br />
FXU Fixed Point Unit<br />
Gb gigabit<br />
GB gigabyte<br />
gcc GNU C Compiler<br />
GPR General Purpose Register<br />
GUI Graphical User Interface<br />
HTTP Hypertext Transfer Protocol<br />
HTTPS Hypertext Transfer<br />
Protocol-Secure<br />
I/O input/output<br />
<strong>IBM</strong> International Business<br />
Machines Corporation<br />
IGESM CISCO Intelligent Gigabit<br />
Ethernet Switch Module<br />
IP Internet Protocol<br />
IPL Initial Program Load (Boot)<br />
ITSO International Technical<br />
Support Organization<br />
KVM keyboard, video and mouse<br />
LAN local area network<br />
LPAR logical partition<br />
LR Link Register<br />
MAC Media Access<br />
Mb megabit<br />
MB megabyte<br />
MFLOPS Million Floating Point<br />
Operations Per Second<br />
MP multiprocessor<br />
NFS Network File System<br />
NIC Network Interface Controller<br />
NIM Network Install Manager<br />
PKT packet<br />
POSIX Portable Operating System for<br />
UNIX<br />
POST Power On Self Test<br />
POWER Performance Optimization<br />
With Enhanced RISC<br />
RAM random access memory<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 159
RHEL Red Hat Enterprise Linux<br />
RISC Reduced Instruction Set<br />
Computer<br />
ROM Read Only Memory<br />
RTC Real Time Clock<br />
SAN storage area network<br />
SCSI small computer systems<br />
interface<br />
SMP Simultaneous Multi-Processor<br />
SOL Serial over LAN<br />
SPOT Shared Product Object Tree<br />
SSH Secure Shell<br />
TCP/IP Transmission Control<br />
Protocol/Internet Protocol<br />
TFTP Trivial File Transfer Protocol<br />
TLB Translation Lookaside Buffer<br />
USB Universal Serial Bus<br />
UXBC UpdateXpress firmware<br />
update scripts for<br />
<strong>BladeCenter</strong><br />
VLAN virtual local area network<br />
VMX Vector Multimedia eXtensions<br />
VPN Virtual Private Network<br />
VXU Vector Processing Unit<br />
WAN Wide Area Network<br />
XER Fixed Point Exception<br />
Register<br />
XML eXtended Markup Language<br />
YAST Yet Another Setup Tool<br />
(SuSE)<br />
160 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Related publications<br />
<strong>IBM</strong> <strong>Redbooks</strong><br />
<strong>The</strong> publications listed in this section are considered particularly suitable for a<br />
more detailed discussion of the topics covered in this redbook.<br />
For information about ordering these publications, see “How to get <strong>IBM</strong><br />
<strong>Redbooks</strong>” on page 164. Note that some of the documents referenced here may<br />
be available in softcopy only.<br />
► Implementing System Management Solutions using <strong>IBM</strong> Director, SG24-6188<br />
► <strong>The</strong> Cutting Edge: <strong>IBM</strong> Eserver <strong>BladeCenter</strong>, REDP-3581<br />
► <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Systems Management, REDP-3582<br />
► Deploying Lotus Domino on <strong>IBM</strong> Eserver <strong>BladeCenter</strong>, REDP-3584<br />
► Deploying Apache on <strong>IBM</strong> Eserver <strong>BladeCenter</strong>, REDP-3588<br />
► Deploying Samba on <strong>IBM</strong> Eserver <strong>BladeCenter</strong>, REDP-3595<br />
► <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Networking Options, REDP-3660<br />
► <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Layer 2-7 Network Switching, REDP-3755<br />
► <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Systems Management with <strong>IBM</strong> Director V4.1<br />
and Remote Deployment Manager 4.1, REDP-3776<br />
► <strong>IBM</strong> Eserver <strong>BladeCenter</strong> Configuration Tips, TIPS-0454<br />
Other publications<br />
<strong>The</strong>se publications are also relevant as further information sources:<br />
► Peterson and Davie, Computer Networks: A Systems Approach, 3rd Edition,<br />
Morgan Kaufmann, 2003, ISBN 1-55-860832-3<br />
► Stevens, TCP/IP Illustrated, Volume 1: <strong>The</strong> Protocols, Addison-Wesley, 1993,<br />
ISBN 0-20-163346-9<br />
► Tanenbaum, Computer Networks, 4th Edition, Prentice Hall, 2002, ISBN<br />
0-13-066102-3<br />
► Tanenbaum, Modern Operating Systems, 2nd Edition, Prentice Hall, 2001,<br />
ISBN 0-13-031358-0<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 161
Online resources<br />
► CSM for Linux: Planning and Installation Guide, SA22-7853<br />
http://publib.boulder.ibm.com/epubs/pdf/am7il103.pdf<br />
► <strong>IBM</strong> Cluster Systems Management for Linux Planning and Installation Guide,<br />
SA22-7873<br />
► <strong>IBM</strong> Cluster Systems Management for AIX5L Planning and Installation Guide,<br />
SA22-7919<br />
► <strong>IBM</strong> Director 4.12 Installation and Configuration Guide, SC09- P290-50<br />
► <strong>IBM</strong> Director 4.0 for <strong>BladeCenter</strong> Products: Systems Management Guide,<br />
SC01-R051-40<br />
► CSM for Linux Planning and Installation Guide, SA22-7853<br />
<strong>The</strong>se Web sites and URLs are also relevant as further information sources:<br />
► Installing AIX 5L on the <strong>IBM</strong> Sserver <strong>BladeCenter</strong> <strong>JS20</strong> whitepaper<br />
http://www.ibm.com/servers/aix/whitepapers/aix_js20.pdf<br />
► <strong>IBM</strong> Eserver Support Web site<br />
http://www.ibm.com/pc/support<br />
► Red Hat Linux<br />
http://www.redhat.com<br />
► SUSE LINUX<br />
http://www.suse.com<br />
► Instructions for installing SuSE from CD<br />
http://www.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm&lndocid=<br />
MIGR-54819<br />
► PowerPC 970 Training<br />
http://www.technonics.com/powerpc.htm<br />
► Linux at <strong>IBM</strong><br />
http://www-1.ibm.com/linux<br />
► Linux Kernel Archives<br />
http://www.kernel.org<br />
► AIX Version 5L Operating System<br />
http://www-1.ibm.com/servers/aix<br />
162 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
► <strong>IBM</strong> Microelectronics Division - PowerPC Technical Documentation<br />
http://www.chips.ibm.com<br />
► <strong>IBM</strong> ^ <strong>BladeCenter</strong> Power Module Upgrade Guidelines Technical<br />
Update<br />
http://www.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-53353<br />
► Myrinet Web site<br />
http://www.myri.com<br />
► <strong>IBM</strong> ^ Cluster 1350 product<br />
http://www.ibm.com/eserver/cluster<br />
► PowerPC Architecture books from the <strong>IBM</strong> DeveloperWorks Web site<br />
http://www.ibm.com/developerworks/eserver/articles/archguide.html<br />
► <strong>IBM</strong> white paper POWER3: Next Generation 64-bit PowerPC Processor<br />
Design<br />
http://www.ibm.com/servers/eserver/pseries/hardware/whitepapers/power3wp.ht<br />
ml<br />
► <strong>IBM</strong> white paper entitled POWER4 System Microarchitecture<br />
http://www.ibm.com/servers/eserver/pseries/hardware/whitepapers/power4.html<br />
► PowerPC Microprocessor Family: AltiVec Technology Programming<br />
Environments Manual<br />
http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/FBFA164F824370F98<br />
7256D6A006F424D<br />
► RETAIN tip H181655<br />
http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-55282<br />
► CSM publication site<br />
http://publib.boulder.ibm.com/clresctr/windows/public/clusterbooks.html<br />
► Service and productivity tools for Linux on POWER<br />
http://techsupport.services.ibm.com/server/lopdiags<br />
► <strong>The</strong> Linux Documentation Project - Kernel HOWTO<br />
http://www.tldp.org<br />
► <strong>The</strong> latest CSM software<br />
http://techsupport.services.ibm.com/server/cluster/fixes/csmfixhome.html<br />
► CSM for Linux: Planning and Installation Guide, SA22-7853<br />
http://publib.boulder.ibm.com/epubs/pdf/am7il103.pdf<br />
Related publications 163
How to get <strong>IBM</strong> <strong>Redbooks</strong><br />
Help from <strong>IBM</strong><br />
You can search for, view, or download <strong>Redbooks</strong>, Redpapers, Hints and Tips,<br />
draft publications and Additional materials, as well as order hardcopy <strong>Redbooks</strong><br />
or CD-ROMs, at this Web site:<br />
ibm.com/redbooks<br />
<strong>IBM</strong> Support and downloads<br />
ibm.com/support<br />
<strong>IBM</strong> Global Services<br />
ibm.com/services<br />
164 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
Index<br />
Symbols<br />
/boot/ppc64 109<br />
/etc/dhcpd.conf 110<br />
/etc/exports 103<br />
/tftpboot 111<br />
Numerics<br />
64-bit computing 32, 34, 36–37, 42, 46<br />
A<br />
activating the open firmware interface 94<br />
Aggressive branch prediction 44–45<br />
AIX operating system 48, 54, 57, 60, 62, 64, 79,<br />
84–85<br />
installing 123<br />
installing with NIM 124–138<br />
Altivec 40<br />
AMD 3DNow! 40<br />
Apple Computer 36<br />
ARP function 56<br />
array 39<br />
array processing 39<br />
AS/400 37<br />
AutoYaST 114<br />
B<br />
blade servers 21<br />
assigning names 77<br />
configuration 76–78<br />
setting the boot sequence 78<br />
Blade Systems Management Processor (BSMP)<br />
15, 22, 26<br />
<strong>BladeCenter</strong><br />
advantages<br />
high availability 4<br />
lower cost 4<br />
SAN optimization 4<br />
server consolidation 4<br />
switch technology 4<br />
chassis 8–10<br />
management module 13<br />
HS20 blade server 21<br />
I/O modules 16–21<br />
Management Module 13–16<br />
power module options 10<br />
power options 10–163<br />
power requirements 5<br />
standard features 9<br />
supported bays 8<br />
versus <strong>IBM</strong> rack server 5<br />
<strong>BladeCenter</strong> T 10<br />
boot sequence 78<br />
Bootstrap protocol (BOOTP) 60<br />
Brocade Enterprise SAN Switch Module 17, 19<br />
Brocade Entry SAN Switch Module 17, 20<br />
Bus Interface Unit (BIU) 45<br />
C<br />
C2 security 48<br />
cache structure 44<br />
Cisco Systems Intelligent Gigabit Ethernet Switch<br />
Module 17–18, 76, 82<br />
Command line interface (CLI) 56, 91<br />
env command 92<br />
Complex instruction set computer (CISC) 30<br />
console for <strong>JS20</strong> blades 87<br />
copycds command 152<br />
copycsmpkgs command 152<br />
count register (CTR) 35<br />
CSM<br />
see <strong>IBM</strong> Cluster Systems Management<br />
csmconfig command 152<br />
D<br />
definenode command 156<br />
DHCP (dynamic host configuration protocol) 60–61<br />
dynamic host configuration protocol (DHCP) 54,<br />
60–61, 70, 110, 149<br />
E<br />
effective to real address translation (ERAT) 44<br />
en0 62<br />
en1 62<br />
env command 92<br />
© Copyright <strong>IBM</strong> Corp. 2005. All rights reserved. 165
eth0 62<br />
eth1 62<br />
F<br />
Fibre Channel Expansion Card 28<br />
firmware 79–87<br />
<strong>JS20</strong> blade server 84<br />
LAN switch I/O module 82<br />
Management Module 80<br />
upgrading under Linux 84<br />
vital product data (VPD) 79<br />
fixed-point exception register (XER) 34<br />
floating-point status and control register (FPSCR)<br />
34<br />
G<br />
general purpose registers (GPRs) 34<br />
Gigabit Ethernet Expansion Card 28<br />
GNU C Compiler (gcc) 49<br />
H<br />
high availability 4<br />
HTTP 16<br />
HTTPS 16<br />
I<br />
I/O modules 16–21, 73–75<br />
configuration 73<br />
<strong>IBM</strong> 4-Port Gigabit Ethernet Switch Module 17, 76,<br />
79, 82<br />
<strong>IBM</strong> AS/400 37<br />
<strong>IBM</strong> Cluster Systems Management (CSM) 16,<br />
50–51, 56, 60, 64, 66, 147–158<br />
obtaining latest software and prerequisites 150<br />
setting up 148<br />
<strong>IBM</strong> Director 16, 56, 63–64, 140<br />
agent 64<br />
installing agent 145<br />
<strong>IBM</strong> General Parallel File System (GPFS) 59<br />
<strong>IBM</strong> Microelectronics Division 46<br />
chip documentation 46<br />
<strong>IBM</strong> Microprocessors<br />
POWER2 31<br />
POWER3 37<br />
POWER4 37<br />
POWER5 38<br />
PowerPC 970 22, 29, 38, 42, 46<br />
166 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong><br />
PowerPC 970FX 22, 29, 38, 42, 46<br />
<strong>IBM</strong> microprocessors<br />
PowerPC 601 36<br />
PowerPC 603 36<br />
PowerPC 604 36<br />
<strong>IBM</strong> rack server 5<br />
<strong>IBM</strong> RS/6000 36<br />
installms command 151<br />
installnode command 158<br />
integrated switch technology 4<br />
Integrated Systems Management Processor (ISMP)<br />
85<br />
Intel Multimedia Extensions (MMX) 40<br />
Intel Streaming SIMD Extensions (SSE) 40<br />
J<br />
<strong>JS20</strong> Blade Server<br />
base features 24<br />
ethernet controller 25<br />
I/O expansion option connectors 25<br />
I/O subsystem 24<br />
IDE controller 25<br />
memory subsystem 24<br />
miscellaneous I/O 26<br />
optional features 27<br />
expansion cards 28<br />
hard disk drives 27<br />
memory modules 27<br />
processor subsystem 24<br />
USB controllers 26<br />
K<br />
keyboard, video monitor and mouse (KVM) 14<br />
KVM (keyboard, video monitor and mouse) 14<br />
L<br />
link register (LR) 35<br />
Linux<br />
compiling a Red Hat kernel 108<br />
compiling a SuSe kernel 108<br />
configuring the sources 102<br />
installing 102<br />
Red Hat sources 102<br />
SuSE sources 104<br />
local area network (LAN) 16<br />
lsnode command 156
M<br />
Management Module 13–16, 70–73<br />
configuration 70–73<br />
console commands 93<br />
default IP address 70<br />
default password 72<br />
default userid 71<br />
firmware 80<br />
mkinitrd command 109<br />
mkzimage_cmdline command 117<br />
Motorola 36<br />
Myrinet 16, 20, 22, 28, 58–59<br />
N<br />
Network File System (NFS) 60<br />
network installation 118<br />
network installation planning 60<br />
network interface selection 62<br />
network planning 54<br />
minimal requirements 54<br />
network services 60<br />
Nortel Networks Layer 2-7 Gigabit Ethernet Switch<br />
Module 17–18, 76, 82<br />
O<br />
obtaining latest CSM software and prerequisites<br />
150<br />
Open Firmware 117<br />
Operating Environment Architecture (OEA) 33<br />
operating system support 48<br />
Optical Pass-thru Module 20<br />
P<br />
PCI<br />
see Peripheral Component Interconnect<br />
performing a network installation 118<br />
performing an unattended installation 116<br />
Peripheral Component Interconnect (PCI) 5, 25,<br />
141<br />
POWER architecture<br />
definition 30<br />
POWER2 31<br />
POWER3 37<br />
POWER4 37<br />
POWER5 38<br />
Power On Self Test (POST) 109<br />
power requirements for <strong>BladeCenter</strong> 5<br />
PowerPC architecture<br />
background 32<br />
definition 30<br />
Operating Environment Architecture (OEA) 33<br />
PowerPC 601 36<br />
PowerPC 603 36<br />
PowerPC 604 36<br />
PowerPC 970 22, 29, 38, 42, 46<br />
PowerPC 970FX 22, 29, 38, 42, 46<br />
User Instruction Set Architecture (UISA) 33<br />
Virtual Environment Architecture (VEA) 33<br />
preparing an unattended installation 111<br />
probemgr command 154<br />
R<br />
rack server 5<br />
rconsole command 157<br />
Red Hat Enterprise Linux 3 48, 121<br />
Red Hat’s Kickstart 112<br />
<strong>Redbooks</strong> Web site 164<br />
Contact us xv<br />
reduced instruction set computer (RISC) 30<br />
Register renaming 44<br />
required network services 60<br />
rpm command 151<br />
rpower command 157<br />
RS64 processors 36<br />
rtas_flash kernel module 84<br />
S<br />
segment lookaside buffer (SLB) 44<br />
Serial over LAN (SoL) 14, 50, 57, 62, 76, 87<br />
components 88<br />
configuring 89<br />
using command line interface 91<br />
server consolidation 4<br />
setting up a CSM management node 148<br />
setting up network installation 61<br />
small computer system interface (SCSI) 5<br />
SoL (Serial over LAN) 14<br />
SSH 16, 89, 91<br />
storage area network (SAN) 16<br />
optimization 4<br />
superscalar features 44<br />
SuSE AutoYaST 114<br />
SUSE LINUX Enterprise Server 9 49, 118<br />
symmetrical multiprocessor (SMP) 36<br />
systemid command 153<br />
Index 167
T<br />
translation lookaside buffer (TLB) 44<br />
Trivial File Transfer Protocol (TFTP) 60, 111<br />
U<br />
unattended installation 111, 116<br />
Universal Manageability Initiative 49<br />
update_flash script 85<br />
User Instruction Set Architecture (UISA) 33<br />
V<br />
vector 39<br />
vector length 38<br />
vector register file (VRF) 41<br />
Vector/SIMD Multimedia eXtension (VMX) 38<br />
Virtual Environment Architecture (VEA) 33<br />
Virtual LAN (VLAN) 15, 17–18, 54, 57, 61, 76, 87,<br />
89<br />
vital product data (VPD) 79<br />
VMX extensions to PowerPC architecture 40<br />
VMX status and control register (VSCR) 41<br />
VPD (vital product data) 79<br />
VRP (vector register file) 41<br />
VSCR (VMX status and control register) 41<br />
X<br />
xinetd daemon 111<br />
Y<br />
Yet Another Setup Tool (YaST) 104<br />
Z<br />
zImage.initrd 116<br />
zImage.initrd file 107, 109<br />
building your own 107–108<br />
168 <strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
<strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> <strong>JS20</strong>
<strong>The</strong> <strong>IBM</strong> Eserver<br />
<strong>BladeCenter</strong> <strong>JS20</strong><br />
Learn about <strong>IBM</strong><br />
Eserver<br />
<strong>BladeCenter</strong> <strong>JS20</strong><br />
and blade server<br />
technology<br />
Understand the<br />
PowerPC 970<br />
microprocessor<br />
architecture<br />
Plan for, install, and<br />
configure<br />
<strong>BladeCenter</strong> <strong>JS20</strong><br />
SG24-6342-01 ISBN 073849061X<br />
Back cover<br />
Blade servers are a relatively new technology. <strong>The</strong>y have<br />
captured industry focus because of their modular design,<br />
which can reduce cost with a more efficient use of valuable<br />
floor space. <strong>The</strong>y offer simplified management, which can<br />
help to speed such tasks as installing, reprovisioning,<br />
updating, and troubleshooting hundreds of blade servers. You<br />
can do all of this remotely using one graphical console with<br />
<strong>IBM</strong> Director systems management tools.<br />
In addition, blade servers provide improved performance by<br />
doubling current rack density. By integrating resources and<br />
sharing key components, costs decrease and availability<br />
increases.<br />
<strong>The</strong> <strong>IBM</strong> Eserver <strong>BladeCenter</strong> boasts innovative modular<br />
technology, leadership density, and availability. It was<br />
designed to help solve a multitude of real-world problems.<br />
This <strong>IBM</strong> Redbook takes an in-depth look at the <strong>IBM</strong><br />
Eserver <strong>BladeCenter</strong> <strong>JS20</strong>. This is a two-way blade server<br />
for applications requiring 64-bit computing. It is ideal for<br />
computer-intensive applications and transactional Internet<br />
servers. This <strong>IBM</strong> Redbook helps you to install, tailor, and<br />
configure the <strong>IBM</strong> Eserver® <strong>BladeCenter</strong> <strong>JS20</strong>.<br />
INTERNATIONAL<br />
TECHNICAL<br />
SUPPORT<br />
ORGANIZATION<br />
®<br />
BUILDING TECHNICAL<br />
INFORMATION BASED ON<br />
PRACTICAL EXPERIENCE<br />
<strong>IBM</strong> <strong>Redbooks</strong> are developed by<br />
the <strong>IBM</strong> International Technical<br />
Support Organization. Experts<br />
from <strong>IBM</strong>, Customers and<br />
Partners from around the world<br />
create timely technical<br />
information based on realistic<br />
scenarios. Specific<br />
recommendations are provided<br />
to help you implement IT<br />
solutions more effectively in<br />
your environment.<br />
For more information:<br />
ibm.com/redbooks