The IBM eServer BladeCenter JS20 - IBM Redbooks

The IBM Eserver
BladeCenter JS20
Learn about IBM Eserver BladeCenter JS20
and blade server technology
Understand the PowerPC 970
microprocessor architecture
Plan for, install, and
configure BladeCenter JS20
ibm.com/redbooks
Ben Gibbs
Omkhar Arasaratnam
Robert Arenburg
Bradley Elkin
Rogeli Grima
James Kelly

International Technical Support Organization
The IBM Eserver BladeCenter JS20
June 2005
SG24-6342-01

Note: Before using this information and the product it supports, read the information in
“Notices” on page xi.
Second Edition (June 2005)
This edition applies to the IBM Eserver BladeCenter JS20, type 8842.
© Copyright International Business Machines Corporation 2005. All rights reserved.
Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP
Schedule Contract with IBM Corp.

Contents
Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Part 1. Introduction to the BladeCenter JS20. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Chapter 1. Introduction to BladeCenter and blade server technology. . . . 3
Chapter 2. Hardware components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 Overview of the BladeCenter infrastructure . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.1 BladeCenter chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.2 BladeCenter power options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.3 Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.1.4 I/O modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.5 Blade servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2 BladeCenter JS20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.1 Base features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.2 Optional features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.3 PowerPC 970 and PowerPC 970FX Microprocessors . . . . . . . . . . . . . . . 29
2.3.1 Review of POWER and PowerPC Architecture . . . . . . . . . . . . . . . . 30
2.3.2 Vector/SIMD Multimedia eXtension . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.3.3 Features of the PowerPC 970 and PowerPC 970FX . . . . . . . . . . . . 42
Chapter 3. Software environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.1 Operating system support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.1.1 AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.1.2 Red Hat Enterprise Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.1.3 SUSE LINUX Enterprise Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2 System management tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2.1 BladeCenter Web interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2.2 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.2.3 IBM Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . 50
© Copyright IBM Corp. 2005. All rights reserved. iii

Chapter 4. Planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.1 Network planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.1.1 Minimal network requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.1.2 High-performance, low-latency network requirements . . . . . . . . . . . 58
4.1.3 Multiple BladeCenter environments . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.2 Operating system installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.2.1 Network installation planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.3 Systems management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.3.1 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.3.2 IBM Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . 64
Part 2. Implementing the BladeCenter JS20. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Chapter 5. Hardware setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.1 Management Module configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.2 LAN Switch I/O Module configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.3 Blade server configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.3.1 Assigning names to blade servers . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.3.2 Setting the boot sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.4 Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5.4.1 Management Module firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.4.2 LAN Switch I/O Module firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.4.3 BladeCenter JS20 firmware (BIOS) . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.4.4 Integrated Systems Management Processor firmware . . . . . . . . . . . 85
5.5 Providing a console for the JS20 blades . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.5.1 Configuring Serial over LAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.5.2 Using Serial over LAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.5.3 Open Firmware interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.5.4 Specifying IP parameters to Open Firmware . . . . . . . . . . . . . . . . . . 96
Chapter 6. Installing Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.1 Installing Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.1.1 Configuring the sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.1.2 The zImage.initrd file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6.2 Configuring BOOTP and TFTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.2.1 BOOTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.2.2 Trivial File Transfer Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6.3 Preparing an unattended install. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6.3.1 Red Hat Kickstart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.3.2 SuSE AutoYaST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.4 Performing an unattended installation. . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.4.1 Open Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.4.2 mkzimage_cmdline: SuSE only. . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.5 Performing a network installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
iv The IBM Eserver BladeCenter JS20

6.5.1 SUSE LINUX Enterprise Server 9. . . . . . . . . . . . . . . . . . . . . . . . . . 118
6.5.2 Red Hat Enterprise Linux 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Chapter 7. Installing AIX on the JS20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
7.1 Minimal NIM installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7.2 Adding resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
7.3 Adding machine information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.4 Preparing the NIM master . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
7.5 Preparing the client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
Chapter 8. System management scenarios . . . . . . . . . . . . . . . . . . . . . . . 139
8.1 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
8.1.1 Setting up an IBM Director Server . . . . . . . . . . . . . . . . . . . . . . . . . 142
8.1.2 Installing an IBM Director Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
8.1.3 Using Director to manage JS20s . . . . . . . . . . . . . . . . . . . . . . . . . . 147
8.2 IBM Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
8.2.1 Setting up a CSM management node . . . . . . . . . . . . . . . . . . . . . . . 148
8.2.2 Installing and managing BladeCenter JS20s using CSM . . . . . . . . 154
Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Contents v

vi The IBM Eserver BladeCenter JS20

Figures
2-1 BladeCenter chassis front view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2-2 BladeCenter chassis rear view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2-3 BladeCenter power domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2-4 KVM Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2-5 Management Module connectors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2-6 BladeCenter JS20 physical layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2-7 BladeCenter JS20 block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2-8 PowerPC logical processing model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2-9 PowerPC registers used by application programs . . . . . . . . . . . . . . . . . 35
2-10 Scalar and vector operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2-11 PowerPC with VMX logical processing model . . . . . . . . . . . . . . . . . . . . 40
2-12 VMX registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2-13 PowerPC 970 internal organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4-1 Network logical view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4-2 Myrinet network infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5-1 Management Module controls and indicators . . . . . . . . . . . . . . . . . . . . 71
5-2 IP address configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5-3 LAN Switch I/O Module IP address . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5-4 LAN Switch I/O Module setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5-5 Blade server naming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5-6 Boot sequence of blades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5-7 Viewing firmware levels on BladeCenter . . . . . . . . . . . . . . . . . . . . . . . . 80
5-8 Updating the Management Module firmware . . . . . . . . . . . . . . . . . . . . . 81
5-9 Firmware upgrade of 4-Port Gb Ethernet Switch Module . . . . . . . . . . . 83
5-10 Blade firmware update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5-11 Serial over LAN components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5-12 Serial over LAN configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5-13 Serial over LAN status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5-14 Management Module’s command line interface . . . . . . . . . . . . . . . . . . 92
5-15 Setting CRLF in Windows 2000 Telnet . . . . . . . . . . . . . . . . . . . . . . . . . 94
5-16 POST checkpoint codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6-1 YaST installation server configuration . . . . . . . . . . . . . . . . . . . . . . . . . 104
6-2 Selecting the distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6-3 Source Configuration pane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
6-4 Completing the setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6-5 redhat-config-kickstart main window . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6-6 AutoYaST module within YaST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6-7 Main AutoYaST window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
© Copyright IBM Corp. 2005. All rights reserved. vii

6-8 Language selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6-9 Main menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6-10 Installation menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
6-11 Source menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
6-12 Protocol menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
6-13 Installation location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7-1 NIM configuration window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7-2 Configuring as a NIM Master . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7-3 Selecting the NIM interface and network name . . . . . . . . . . . . . . . . . . 125
7-4 Choosing to initialize minimal NIM environment . . . . . . . . . . . . . . . . . 126
7-5 Finalizing the NIM environment settings . . . . . . . . . . . . . . . . . . . . . . . 126
7-6 lpp_source and SPOT configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 127
7-7 AIX source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7-8 Default resource settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7-9 Completing the resource installation . . . . . . . . . . . . . . . . . . . . . . . . . . 129
7-10 Adding new machine information to NIM . . . . . . . . . . . . . . . . . . . . . . . 130
7-11 Host name of new machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
7-12 Entering machine-specific information . . . . . . . . . . . . . . . . . . . . . . . . . 132
7-13 Entering MAC address information . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7-14 Installing an operating system from NIM . . . . . . . . . . . . . . . . . . . . . . . 134
7-15 Setting BOS preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
7-16 Installation language panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7-17 Operating system selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7-18 Installation and language options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7-19 Kernel options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
8-1 Selecting the type of database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
8-2 Database connection information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
8-3 Setting Director server encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
8-4 Installing the Director Agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
8-5 Configuring Director Agent security . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
viii The IBM Eserver BladeCenter JS20

Tables
2-1 Power domain A minimum power requirements . . . . . . . . . . . . . . . . . . 12
2-2 Power domain B minimum power requirements . . . . . . . . . . . . . . . . . . 13
2-3 Physical attributes of PowerPC 970 and PowerPC 970FX . . . . . . . . . . 42
5-1 BladeCenter and BladeCenter JS20 firmware components . . . . . . . . . 79
5-2 Management Module commands for SoL . . . . . . . . . . . . . . . . . . . . . . . 93
5-3 Network configuration information worksheet . . . . . . . . . . . . . . . . . . . . 96
© Copyright IBM Corp. 2005. All rights reserved. ix

x The IBM Eserver BladeCenter JS20

Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area.
Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product, program, or service that
does not infringe any IBM intellectual property right may be used instead. However, it is the user's
responsibility to evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document.
The furnishing of this document does not give you any license to these patents. You can send license
inquiries, in writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such provisions
are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES
THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer
of express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may
make improvements and/or changes in the product(s) and/or the program(s) described in this publication at
any time without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without
incurring any obligation to you.
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm
the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on
the capabilities of non-IBM products should be addressed to the suppliers of those products.
This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrates programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the
sample programs are written. These examples have not been thoroughly tested under all conditions. IBM,
therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,
modify, and distribute these sample programs in any form without payment to IBM for the purposes of
developing, using, marketing, or distributing application programs conforming to IBM's application
programming interfaces.
© Copyright IBM Corp. 2005. All rights reserved. xi

Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
Eserver®
Eserver®
developerWorks®
eServer
ibm.com®
iSeries
pSeries®
xSeries®
AIX 5L
AIX®
AS/400®
xii The IBM Eserver BladeCenter JS20
BladeCenter
Domino®
IBM®
Lotus®
PowerPC Architecture
PowerPC 601®
PowerPC 603
PowerPC 604
PowerPC®
POWER
POWER2
The following terms are trademarks of other companies:
POWER3
POWER4
POWER4+
POWER5
Redbooks
Redbooks (logo)
RETAIN®
RS/6000®
ServeRAID
System/360
3090
Intel, Intel Inside (logos), MMX, and Pentium are trademarks of Intel Corporation in the United States, other
countries, or both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the
United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other company, product, and service names may be trademarks or service marks of others.

Preface
Blade servers are a relatively new technology. They have captured industry focus
because of their modular design, which can reduce cost with a more efficient use
of valuable floor space. They offer simplified management, which can help to
speed such tasks as installing, reprovisioning, updating, and troubleshooting
hundreds of blade servers. You can do all of this remotely using one graphical
console with IBM® Director systems management tools.
In addition, blade servers provide improved performance by doubling current rack
density. By integrating resources and sharing key components, costs decrease
and availability increases.
The IBM Eserver® BladeCenter boasts innovative modular technology,
leadership density, and availability. It was designed to help solve a multitude of
real-world problems.
This IBM Redbook takes an in-depth look at the IBM Eserver BladeCenter
JS20. This is a two-way blade server for applications requiring 64-bit computing.
It is ideal for computer-intensive applications and transactional Internet servers.
This IBM Redbook helps you to install, tailor, and configure the IBM Eserver
BladeCenter JS20.
The team that wrote this redbook
This redbook was produced by a team of specialists from around the world
working at the International Technical Support Organization (ITSO), Austin
Center.
Ben Gibbs is a Senior Consulting Engineer with Technonics, Inc.
(http://www.technonics.com) in Austin, Texas. He has over 20 years of
experience with UNIX®-based operating systems. He started working with the
AIX® operating system in November of 1989. His areas of expertise include
performance analysis and tuning, operating system internals, and device driver
development for the AIX operating system. He is also an IBM Learning Services
instructor for advanced AIX classes. He was the project leader for this IBM
Redbook.
Omkhar Arasaraknum is a Team Leader with IBM Global Services in Canada.
He has worked with Linux since 1998 and uses SuSE, Red Hat, and Gentoo.
Omkhar is the Linux technical lead within his Service Delivery Center and has
© Copyright IBM Corp. 2005. All rights reserved. xiii

worked actively with Linux adoption within IBM. Omkhar has also written IBM
Redpapers about Microsoft® Windows® to Linux migrations for IBM.
Robert Arenburg is a Senior Technical Consultant in the Solutions Enablement
organization in the IBM Systems and Technology Group and is located in Austin,
Texas. He has a PhD in Engineering Mechanics from Virginia Tech. He has
worked at IBM for 13 years. His areas of expertise include high performance
computing, performance, capacity planning, 3D graphics, solid mechanics,
computational, and finite element methods.
Bradley Elkin is a Senior Software Engineer for IBM U.S. He holds a PhD in
Chemical Engineering from the University of Pennsylvania and has 17 years of
experience in high performance computing. His areas of expertise include
applications in computational fluid mechanics, computational chemistry, and
bioinformatics. He has written several articles for IBM ^ Development
Domain.
Rogeli Grima is a research staff member at the CEPBA IBM Research Institute
in Spain. He has three years of experience in applied mathematics. He has also
collaborated on the development of BladeCenter JS20 solution for
bioinformatics.
James Kelly is a Systems Architect in IBM Australia specializing in deep
computing. He has 17 years of experience in system engineering and technical
sales support. His current areas of expertise include the solution design,
benchmarking, and installation of server clusters running either AIX or Linux, to
support the installation of high performance computing applications.
Thanks to the following people for their contributions to this project:
ITSO, Austin Center
Trina Bunting
IBM Dallas
Luciano Chavez
Matt Davis
Martin Gramlich
Jon Mason
Doug W Oliver
Brian Rzycki
IBM Austin
Anton Blanchard
IBM OzLabs, Canberra, Australia
xiv The IBM Eserver BladeCenter JS20

Janet Ellsworth
IBM Poughkeepsie, New York
Become a published author
Join us for a two- to six-week residency program! Help write an IBM Redbook
dealing with specific products or solutions, while getting hands-on experience
with leading-edge technologies. You'll team with IBM technical professionals,
Business Partners and/or customers.
Your efforts will help increase product acceptance and customer satisfaction. As
a bonus, you'll develop a network of contacts in IBM development labs, and
increase your productivity and marketability.
Find out more about the residency program, browse the residency index, and
apply online at:
Comments welcome
ibm.com/redbooks/residencies.html
Your comments are important to us!
We want our Redbooks to be as helpful as possible. Send us your comments
about this or other Redbooks in one of the following ways:
► Use the online Contact us review redbook form found at:
ibm.com/redbooks
► Send your comments in an Internet note to:
redbook@us.ibm.com
► Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. JN9B Building 905 Internal Zip 9053D003
11501 Burnet Road
Austin, Texas 78758-3493
Preface xv

xvi The IBM Eserver BladeCenter JS20

Part 1 Introduction
Part 1
to the
BladeCenter JS20
This part introduces and describes the hardware and software components of
the IBM Eserver BladeCenter JS20. It also presents planning considerations.
© Copyright IBM Corp. 2005. All rights reserved. 1

2 The IBM Eserver BladeCenter JS20

1
Chapter 1. Introduction to BladeCenter
and blade server technology
This chapter presents an overview of IBM Eserver BladeCenter and blade
server technology. It also covers the hardware for the chassis and the blades.
The term blade server refers to a chassis that can hold a number of
hot-swappable devices called blades. Blades come in two varieties: server blades
and option blades.
A server blade is an independent server that contains one or more processors
and associated memory, disk storage, and network controllers. It runs its own
operating system and applications. Each server blade within a system chassis
slides into a blade bay2. It plugs into a midplane or backplane to share common
infrastructure components. These components may include power supplies, fans,
CD-ROM, and floppy drives, Ethernet and Fibre Channel switches, and system
ports.
Option blades may be sharable by server blades. Option blades provide
additional features, such as controllers for external I/O or disk arrays, power
supplies, and so on.
For organizations that seek server consolidation, BladeCenter centralizes
servers for increased flexibility, ease of maintenance, reduced cost, and
streamlined human resources. Companies that need to install new applications
© Copyright IBM Corp. 2005. All rights reserved. 3

for e-commerce and On Demand Business can achieve speed while ensuring
flexibility, scalability, and availability. For such enterprise requirements as
file-and-print and collaboration, BladeCenter is designed to offer reliability,
flexibility for growth, and cost effectiveness. And clients with compute-intensive
applications that need highly available clustering can use the BladeCenter to
help achieve high degrees of scalability and performance.
Benefits of the IBM Eserver BladeCenter family
The IBM Eserver BladeCenter family of products features a modular design that
integrates multiple computing resources into a cost-effective, high-density
enclosure for a platform that provides:
► Lower cost in certain configurations: Due to efficiencies in power usage, heat
emissions, and data center floor space utilization, when a configuration of
other servers exceeds one rack, it is often less expensive over a multi-year
period to use a single rack of BladeCenter chassis and BladeCenter JS20s.
► Physical server consolidation: A BladeCenter server is an ideal way to
replace many uniprocessor or 2-way servers to save space. 14U of rack
space for 14 1U servers can be replaced with one 7U BladeCenter chassis.
Additional rack space that is normally taken for Ethernet, Fibre Channel, and
other switches is eliminated by the integrated switches in the BladeCenter
chassis.
► High availability: The BladeCenter chassis offers redundant and hot-swap
components that can prevent failure of the chassis or blade servers when one
of those components fail. The chassis has redundant, hot-swap cooling,
power, midplane features, management, and I/O switches. It also has a
hot-swap media tray, with CD-ROM and floppy drive, that can be removed
and serviced while all blade servers are operating.
► Integrated switch technology: All service is from the front and rear of
BladeCenter. There is no need to slide the chassis out of the rack and remove
the top cover for service. Also, numerous cables are eliminated, reducing
both cabling cost and servicer or administrator time.
► More integrated systems management features: The BladeCenter chassis
includes a management module. This module eliminates the need for
individual management adapters, such as the Remote Supervisor Adapter
I or II, in each blade server for remote control. It also eliminates the need for
RS485 interconnect cabling between the blade servers.
► SAN optimization: A Fibre Channel switch module installed in a BladeCenter
chassis provides storage area network (SAN) access to all blade servers in
the chassis without internal cabling.
4 The IBM Eserver BladeCenter JS20

BladeCenter versus an IBM rack server
BladeCenter is an ideal solution for certain environments. In other environments,
an IBM rack server may be a better fit:
► Need for a small number of servers: A BladeCenter chassis is required for one
blade to be cost effective compared to some stand-alone servers. Therefore,
a chassis should be full or nearly full.
► Need for existing Peripheral Component Interconnect (PCI) adapters:
BladeCenter does not include adapter slots as shipped. An optional I/O
Expansion Card feature offers one PCI slot per blade, limiting the blade to one
integrated development environment (IDE) drive. The expansion feature
supports PCI cards designed for BladeCenter. Therefore, a mix of
BladeCenter and traditional rack-optimized servers may be appropriate.
► Need for large internal storage or external SCSI storage: BladeCenter
supports a maximum of 80 GB of IDE or 146.8 GB of SCSI storage internally.
While there is no provision for external SCSI storage, there is an external
Fibre Channel SAN storage option. Using the optional internal SCSI storage
feature doubles the space requirement of the blade, cutting in half the number
of blade servers that can be installed in a chassis.
For large internal storage, the IBM Eserver xSeries® 345 is capable of
holding up to 880.8 GB of hot-swap SCSI storage. All rack-mounted xSeries
servers can support, either natively or via an optional ServeRAID adapter,
external SCSI arrays.
► Need for re-installation or repurposing at end of original: Stand-alone
uniprocessor and 2-way servers can be distributed individually for use as
departmental file/print servers and other low-horsepower uses. Because
blade servers cannot be used without a chassis, the entire
chassis-and-blades combo needs to stay together. The original chassis may
be useful if older blade servers are discarded and newer, faster ones take
their places in the chassis.
► Power requirements: The data center is wired for 110-120 V power.
BladeCenter requires 220-240 V power.
Learn more about IBM Eserver BladeCenter and its components in the IBM
Redpaper The Cutting Edge: IBM Eserver BladeCenter, REDP-3581.
The following chapters address the descriptions and considerations of the
hardware and software components.
Chapter 1. Introduction to BladeCenter and blade server technology 5

6 The IBM Eserver BladeCenter JS20

Chapter 2. Hardware components
2
This chapter describes the major hardware components associated with the IBM
^ BladeCenter JS20. It begins by reviewing the core BladeCenter
infrastructure. Then it provides a detailed description of the BladeCenter JS20
and the range of options that you can order from IBM. Finally, this chapter
reviews the IBM PowerPC® 970 RISC Microprocessor and IBM PowerPC 970FX
RISC Microprocessor that are used as main processors on the BladeCenter
JS20.
© Copyright IBM Corp. 2005. All rights reserved. 7

2.1 Overview of the BladeCenter infrastructure
The BladeCenter JS20 is designed to be installed in the IBM ^
BladeCenter Type 8677. This section reviews the BladeCenter infrastructure that
is relevant to the installation of the BladeCenter JS20.
2.1.1 BladeCenter chassis
The core component of the BladeCenter infrastructure is the BladeCenter
chassis. Figure 2-1 illustrates the front view of the BladeCenter chassis. The
BladeCenter chassis is designed to be installed in an industry standard 19-inch
rack. Each BladeCenter chassis occupies seven rack units. You can install up to
six BladeCenter chassis in a single 42U rack such as the IBM NetBAY42
Enterprise Rack.
In the front view of the BladeCenter chassis, you see these standard features:
► Fourteen bays where you can install blade servers
► A media bay containing one CD-ROM drive, one diskette drive, and a
Universal Serial Bus (USB) port that can be dynamically assigned to any
single blade server in the BladeCenter chassis
Figure 2-1 BladeCenter chassis front view
8 The IBM Eserver BladeCenter JS20

Figure 2-2 illustrates the rear view of the BladeCenter chassis. In the rear of the
BladeCenter chassis, you see these standard features:
► One management module
► One bay where you can install an optional redundant Management Module
► Four bays where you can install optional input/output (I/O) modules
► A redundant pair of power supply modules
► Two bays where you can install an additional pair of redundant power supply
modules
► A redundant pair of blowers
Figure 2-2 BladeCenter chassis rear view
Chapter 2. Hardware components 9

Attention: Nonredundant power is not supported in BladeCenter products.
Power modules must always be present in power bays 1 and 2. When any
blade server or option is in blade bays 7 through 14, power modules must be
present in all four power-module bays.
If a power module fails or an ac power failure occurs, BladeCenter units
configured for redundant power operation operate in a nonredundant mode, and
the blower modules runs at full speed. You must replace the failing power module
or restore ac power as soon as possible to regain redundant power operation and
to reset the blower modules to their normal operating speed.
As of the date of this publication, these four BladeCenter power-module options
are supported:
► IBM BladeCenter 1200W Power Supply Module (part number 48P7052)
► IBM BladeCenter 1200W to 1400W Power Supply Upgrade Kit (part number
90P0197)
► IBM BladeCenter 1800W Power Supply Module (part number 13N0570)
► IBM BladeCenter 2000W Power Supply Module (part number 26K4816)
Not visible in either view of the BladeCenter chassis are the redundant pair of
midplanes in the center of the BladeCenter chassis that interconnect the blade
servers installed in the front, and other modules that are installed in the rear of
the BladeCenter chassis. The midplanes support the hot-plugging of blade
servers and the other BladeCenter modules.
Note: IBM offers a variant of the BladeCenter called the BladeCenter T. The
BladeCenter T is designed to meet the special needs of the
telecommunications industry and has been tested for NEBS compliance.
Since BladeCenter JS20 is not supported in the BladeCenter T at this time, we
do not describe BladeCenter T and its associated features in this book.
2.1.2 BladeCenter power options
The standard redundant power supplies are installed in power bays 1 and 2 of
the BladeCenter chassis, and provide power to the first six blade server bays.
This is known as power domain A.
To install blade servers in the remaining bays 7 through 14, you must install an
additional pair of redundant power supply modules in power bays 3 and 4. This is
known as power domain B. Figure 2-3 shows these two power domains.
10 The IBM Eserver BladeCenter JS20

Figure 2-3 BladeCenter power domains
Chapter 2. Hardware components 11

Attention: Power supply modules must always be installed in pairs. You can
have either two power supply modules (in power bays 1 and 2) or four power
supply modules (in power bays 1 through 4). Both power supply modules in a
redundant pair should have the same capacity.
When setting up the power requirements for the BladeCenter, remember the
following criteria:
► Install the power modules in pairs in a domain. They must match each other in
capacity (wattage, amperage, and so on).
► A power domain operating above the capacity of a single power module
results in a nonredundant power condition.
► In a pair of power modules, a power module that is not connected to a
200–240 volt ac power source results in a nonredundant power condition.
► To provide true redundant power, connect BladeCenter power modules 1
and 3 to a different 200–240 volt ac power source than to power modules 2
and 4.
► Connect an installed power module to an ac power source. The module must
not be used as a filler.
Attention: To maintain proper system cooling, each unoccupied blade bay
must contain a filler blade as well as each unoccupied I/O,
management-module, or power-module bay. An installed power module must
be powered, but must not be used as a filler.
Table 2-1 shows the minimum power requirements for power domain A.
Table 2-1 Power domain A minimum power requirements
Sum of JS20 power units Minimum power module required
Less than 7.4 1200 W (labeled 7.5A)
7.4 up to less than 9.0 1400 W (labeled 9A)
9.0 up to less than 10.0 1800 W (labeled 12A)
10.0 or greater 2000 W (labeled 13.5A)
12 The IBM Eserver BladeCenter JS20

Table 2-2 shows the minimum power requirements for power domain B.
Table 2-2 Power domain B minimum power requirements
Sum of JS20 power units Minimum power module required
Less than 9.9 1200 W (labeled 7.5A)
9.9 up to less than 11.5 1400 W (labeled 9A)
11.5 up to less than 13.4 1800 W (labeled 12A)
13.4 or greater 2000 W (labeled 13.5A)
Each BladeCenter JS20 (1.6 GHz/800 MHz 8842-21X) requires 1.5 power units
for two installed microprocessors. For example, if you fully populate all six bays,
the total power units is 9.0 (6 bays times 1.5). According to Table 2-1, you need a
minimum of the 1800 W power module.
For other configurations, refer to the latest version of IBM ^ BladeCenter
Power Module Upgrade Guidelines Technical Update, found at:
http://www.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-53353
Also, you can use the following steps to obtain the BladeCenter Planning and
Installation Guide:
1. Go to:
http://www.ibm.com/pc/support/
2. In the Browse by product section, click Servers, and then select the brand
Servers and the family BladeCenter.
3. Click Continue.
4. From the View by document type menu, click Online Publications.
2.1.3 Management Module
Each BladeCenter chassis comes standard with a Management Module, as
shown in Figure 2-4. You can also install a redundant Management Module by
ordering the IBM KVM/Redundant Management Module Option (P/N 48P7055).
When you have two Management Modules, only one is active. The other module
is in standby mode until it is switched to become the active Management Module.
Chapter 2. Hardware components 13

Figure 2-4 KVM Management Module
The Management Module provides the following three major functions:
► Console switching and remote graphical console for blade servers that use a
keyboard, video monitor, and mouse (KVM) for their console
► Serial over local area network (LAN) (SoL) remote text consoles for blade
servers that support SoL consoles (includes the BladeCenter JS20)
► Service processor for the entire BladeCenter
Important: The console switching and remote graphical console functionality
of the Management Module is not used by the BladeCenter JS20 because it
does not support KVM consoles. You can still use the console switching and
remote graphical console functions if you have other blade servers installed in
the same BladeCenter chassis that support KVM consoles.
You use the service processor functionality of the Management Module to
perform many different tasks to:
► Initially configure BladeCenter
► Define and manage the users who can remotely access the Management
Module
► Monitor the BladeCenter and reporting abnormal conditions to system
managers either via e-mail or Simple Network Management Protocol (SNMP)
events sent to management systems
► Remotely power on and off individual blade servers and I/O modules
14 The IBM Eserver BladeCenter JS20

► Remotely switch the CD-ROM, diskette, and USB interfaces in the media bay
of the BladeCenter chassis to a particular blade server
► Maintain the firmware embedded in various BladeCenter components
The Management Module is connected to each blade server and I/O module via
management buses embedded in the BladeCenter chassis midplanes. The
Management Module uses these connections to provide service processor
functionality in conjunction with the Blade Systems Management Processor
(BSMP) on each blade server.
When using the SoL remote text console function, the Management Module also
uses a virtual LAN (VLAN) provided by the LAN switch module installed in I/O
module bay 1 to communicate with each blade server. This VLAN is not
accessible externally.
Each Management Module is equipped with several connectors, as illustrated in
Figure 2-5.
Figure 2-5 Management Module connectors
Chapter 2. Hardware components 15

2.1.4 I/O modules
You use the keyboard, video, and mouse connectors to attach a local KVM
console that can be switched to any blade server that supports KVM consoles.
You use the 10/100BaseT Ethernet connector to attach the Management Module
to an external Ethernet network. You use this interface to access the remote
graphical console, SoL consoles, and the service processor functionality of the
Management Module.
You can access the various functions of the Management Module via the
Ethernet interface using several different methods:
► Using a graphical interface through a Web browser (HTTP or HTTPS)
► Using a command line interface through a Telnet or SSH client
► Using system management software such as IBM Director or IBM Cluster
Systems Management
We demonstrate these methods in Chapter 5, “Hardware setup” on page 69, and
in Chapter 8, “System management scenarios” on page 139.
I/O modules enable connectivity between blade servers within the same chassis,
and between blade servers and the outside world. You can install up to four I/O
modules in a BladeCenter chassis, one in each I/O module bay.
The available I/O modules can be used to provide connectivity to three different
types of network:
► Local area networks
► Storage area networks (SANs)
► High-bandwidth low-latency networks used in clustering (Myrinet)
Most I/O modules support connectivity to only one type of network. The
exception is the pass-through module, which can be used with all three types of
networks.
Each I/O module is connected to every blade server via the BladeCenter chassis
midplanes. Each single width blade server can have up to four interfaces, one to
each I/O module bay. Double-width blade servers can have up to eight interfaces,
two to each I/O module bay.
Each blade server must have a compatible interface to communicate with an I/O
module. Every blade server that is currently available has standard Gigabit
Ethernet interfaces that are connected to I/O module bays 1 and 2. Therefore,
you can use only a LAN Switch or Pass-Thru I/O Module in these I/O module
bays.
16 The IBM Eserver BladeCenter JS20

Restriction: You must install one of the LAN switch modules in I/O module
bay 1 when using the SoL remote text console function. This function depends
on a private VLAN provided by the LAN switch to function correctly.
Every single-width blade server currently available supports the installation of an
optional expansion card that connects to I/O module bay 3 and 4. Double-width
blade servers support two optional expansion cards. If you install an expansion
card, it must be compatible with any I/O modules installed in I/O module bays
3 and 4.
You can order the following LAN Switch I/O Modules:
► P/N 13N0568: 4 port Gigabit Ethernet Switch Module
► P/N 13N2281: CISCO Systems Intelligent Gigabit Ethernet Switch Module
► P/N 73P9057: Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module
You can order the following SAN switch I/O modules:
► P/N 48P7062: 2-Port Fibre Channel Switch Module
► P/N 26K5601: Brocade Entry SAN Switch Module
► P/N 90P0165: Brocade Enterprise SAN Switch Module
For the Pass-Thru I/O Module, you can order Optical Pass-thru Module (P/N
02R9080).
We describe each of these modules in more detail in the sections that follow.
IBM 4-Port Gigabit Ethernet Switch Module
The IBM 4-Port Gigabit Ethernet Switch Module is a layer 2 LAN switch that
provides four external 10/100/1000BaseT interfaces for connecting to LANs that
are external to the BladeCenter chassis. Internally the switch module is designed
to connect to a single Gigabit Ethernet interface on each installed single-width
blade server, or two such interfaces on each installed double-width blade server.
The IBM 4-Port Gigabit Ethernet Switch Module is also connected via a Fast
Ethernet interface to each installed Management Module. This is used to
remotely access the switch module via the Management Module’s 10/100BaseT
Ethernet interface. You can use this capability to support initial configuration and
management of the switch module.
Major features of the 4-Port Gigabit Ethernet Switch Module include:
► VLAN support based on either port or 802.1Q tagging
► Link aggregation of multiple external interfaces to provide increased
bandwidth to external networks
Chapter 2. Hardware components 17

Note: The IBM 4-Port Gb Ethernet Switch Module for BladeCenter may be
installed in your BladeCenter unit and you must replace the CD-ROM media
tray in your BladeCenter unit. In this case, you must also update the
microcode in the Ethernet switch module.
CISCO Systems Intelligent Gigabit Ethernet Switch Module
The CISCO Systems Intelligent Gigabit Ethernet Switch Module (IGESM) is a
layer 2-3 LAN switch that provides four external 10/100/1000BaseT interfaces for
connecting to LANs external to the BladeCenter chassis. Internally the switch
module is designed to connect to a single Gigabit Ethernet interface on each
installed single width blade server, or two such interfaces on each installed
double-width blade server.
The IGESM is also connected via a Fast Ethernet interface to each installed
Management Module. This is used to remotely access the switch module via the
Management Module’s 10/100BaseT Ethernet interface. You can use this
capability to support initial configuration and management of the switch module.
The IGESM uses CISCO IOS. Use the IGESM when you require the advanced
functionality of CISCO IOS or maximum compatibility with external networks that
constructed using switches running CISCO IOS.
Note: To use the SoL remote text console function with the IGESM, you must
manually create the required VLAN using the procedures in the CISCO
Systems Intelligent Gigabit Ethernet Switch Module - Installation Guide.
Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module
The Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module is a layer 2-7
LAN switch that provides four external 10/100/1000BaseT interfaces for
connecting to LANs external to the BladeCenter chassis. Internally the switch
module is designed to connect to a single Gigabit Ethernet interface on each
installed single width blade server, or two such interfaces on each installed
double-width blade server.
The Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module is also
connected via a Fast Ethernet interface to each installed Management Module.
This is used to remotely access the switch module via the Management Module’s
10/100BaseT Ethernet interface. You can use this capability to support initial
configuration and management of the switch module.
18 The IBM Eserver BladeCenter JS20

The Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module provides several
advanced networking features including:
► Rich layer 2 and layer 3 switching capabilities
► Virtual server load balancing (layer 4 switching)
► Content aware load balancing (layer 7 switching)
► High availability clustering of switch modules in the same or different chassis
You can learn more about the Nortel Networks Layer 2-7 Gigabit Ethernet Switch
Module in the IBM Redpaper IBM Eserver BladeCenter Layer 2-7 Network
Switching, REDP-3755.
2-Port Fibre Channel Switch Module
The 2-Port Fibre Channel Switch Module is a Fibre Channel SAN switch that
provides two external 2 Gbps Fibre Channel interfaces for connecting to SANs
external to the BladeCenter chassis. Internally the switch module is designed to
connect to a single 2 Gbps Fibre Channel interface on each installed blade
server that is equipped with a Fibre Channel expansion card.
The 2-Port Fibre Channel Switch Module is also connected via a Fast Ethernet
interface to each installed Management Module. This is used to remotely access
the switch module via the Management Module’s 10/100BaseT Ethernet
interface. You can use this capability to support initial configuration and
management of the switch module.
The 2-Port Fibre Channel Switch Module can be installed only in I/O module
bays 3 or 4. If you install a 2-Port Fibre Channel Switch Module, any blade
servers that have expansion cards must use a Fibre Channel expansion card.
Brocade Enterprise SAN Switch Module
The Brocade Enterprise SAN Switch Module is a Fibre Channel SAN switch that
provides two external 1 Gbps or 2 Gbps (autosense) Fibre Channel interfaces for
connecting to SANs external to the BladeCenter chassis. Internally the switch
module is designed to connect to a single 2 Gbps Fibre Channel interface on
each installed blade server that is equipped with a Fibre Channel expansion
card.
The Brocade Enterprise SAN Switch Module is also connected via a Fast
Ethernet interface to each installed Management Module. This is used to
remotely access the switch module via the Management Module’s 10/100BaseT
Ethernet interface. You can use this capability to support initial configuration and
management of the switch module.
Chapter 2. Hardware components 19

The Brocade Enterprise SAN Module can be installed only in I/O module bays
3 or 4. If you install a 2-Port Fibre Channel Switch Module, any blade servers that
have expansion cards must use a Fibre Channel expansion card.
Use the Brocade Enterprise SAN Switch Module when you require maximum
compatibility with external SAN fabrics constructed using Brocade-based
switches. The Brocade Enterprise SAN Switch Module is functionality equivalent
to the Brocade Silkworm 3900 SAN switch.
Brocade Entry SAN Switch Module
The Brocade Entry SAN Switch Module is almost identical to the Brocade
Enterprise SAN Switch Module described earlier. However, it is limited to
participating in SAN fabrics that contain at most two SAN switches.
Optical Pass-thru Module
The Optical Pass-thru Module provides the capability to access one of the
network interfaces on each blade server via an externally accessible optical
interface. You can use this optical interface to connect the blade servers to
external switches that support compatible optical interfaces, including:
► LAN switches
► SAN switches
► Myrinet switches
Each Optical Pass-thru Module has four multi-port optical connectors. The
connectors are designed for special break-out cables that provide either four SC
connectors (P/N 73P5992) or four LC connectors (P/N 73P6033) per multi-port
optical connector. You can connect to a maximum of 14 blade server network
interfaces using four of these cables (two of the connectors are unused on the
fourth cable).
The type of external switch that you connect to the Optical Pass-thru Module
must be compatible with the network interface on the blade servers that are
connected to the I/O module bay where the Optical Pass-thru Module is installed.
If you install an Optical Pass-thru Module in I/O module bay 1 or 2, you must
connect the optical interfaces to a Gigabit Ethernet switch with 1000Base-SX
interfaces.
Restriction: When a BladeCenter chassis contains a BladeCenter JS20,
always use a LAN switch module in I/O module bay 1 to support the SoL
remote text console function. This is because there is no other mechanism for
providing a console on a BladeCenter JS20.
20 The IBM Eserver BladeCenter JS20

2.1.5 Blade servers
If you install an Optical Pass-thru Module in I/O module bay 3 or 4, you must
connect the optical interfaces to an external switch that is compatible with the
type of expansion cards installed on blade servers.
If you install a Myrinet expansion card on any blade server, you must install an
Optical Pass-thru Module in I/O module bay 4 to enable the Myrinet interface on
the expansion card to be connected to an external Myrinet switch. This is
because currently no Myrinet switch modules are available for the BladeCenter.
IBM offers a variety of blade servers that you can install in a BladeCenter
chassis.
► IBM ^ BladeCenter JS20 Type 8842 Blade Server
► IBM ^ BladeCenter HS20 Type 8678 Blade Server
► IBM ^ BladeCenter HS20 Type 8832 Blade Server
► IBM ^ BladeCenter HS40 Type 8839 Blade Server
Because the BladeCenter JS20 is the subject of this book, we describe it in
greater detail in 2.2, “BladeCenter JS20” on page 22.
The following sections briefly introduce the other blade servers. You can install
any combination of blade servers in the same BladeCenter chassis subject to
available power supply module capacity.
HS20 Blade Server
The HS20 Blade Server is a single-width blade server that contains either one or
two Intel® Xeon DP processors and up to 8 GB of memory. Up to 14 HS20 Blade
Servers can be installed in each BladeCenter chassis.
The original HS20 Type 8678 Blade Server used a 400 MHz front side bus (FSB)
and supports a range of processor speeds from 2.0 GHz to 2.8 GHz. The HS20
Type 8678 Blade Server is now withdrawn from marketing and has been replaced
by the HS20 Type 8832 Blade Server.
The HS20 Type 8832 Blade Server uses a 533 MHz FSB and supports a range
of processor speeds from 2.4 Ghz to 3.2 GHz.
HS40 Blade Server
The HS40 Blade Server is a double-width blade server that contains from one to
four Intel Xeon MP processors and up to 16 GB of memory. Up to seven HS40
Blade Servers can be installed in each BladeCenter chassis.
Chapter 2. Hardware components 21

2.2 BladeCenter JS20
The BladeCenter JS20 is a single-width blade server that offers these features:
► Two PowerPC 970 or PowerPC 970FX microprocessors
► Up to 4 GB of RAM
► Up to two integrated development environment (IDE) hard disk drives
mounted on the blade server
► Two integrated Gigabit Ethernet interfaces
► Expansion option connectors for an expansion card option to provide either:
– Two additional Gigabit Ethernet interfaces
– Two 2 Gbps Fibre Channel interfaces
– One Myrinet interface
► Blade Systems Management Processor
Figure 2-6 illustrates the physical layout of the BladeCenter JS20.
I/O expansion option
I/O expansion option
connector (J18)
connector (J22)
Four Memory DIMMS
Secondary IDE (J2) Primary IDE (J1) Battery
(shown with attached disk)
Figure 2-6 BladeCenter JS20 physical layout
22 The IBM Eserver BladeCenter JS20
Microprocessor 0
and heat sink (U87)
Microprocessor 1
and heat sink (U41)

The architecture of the BladeCenter JS20 is illustrated by the diagram in
Figure 2-7.
PowerPC
970
or
970FX
6.4 GBps
or
8.8 GBps
PowerPC
970
or
970FX
Memory Controller and Host I/O Bridge
(Northbridge)
HyperTransport PCI-X Tunnel
HyperTransport I/O Hub
IDE Controller
IDE
Disk
IDE
Disk
1.6 GBps
800 MBps
6.4 GBps
or
8.8 GBps
5.3 GBps
Real Time
Clock
1.06 GBps
1.06 GBps
USB
Controllers
Super I/O
Figure 2-7 BladeCenter JS20 block diagram
UART
BSMP
PC2700 ECC
Memory Modules
Gigabit
Ethernet
Controller
Expansion Card
Boot
FLASH
NVRAM
Chassis
Midplanes
The BladeCenter JS20 includes both base features and a range of optional
features. We describe each of these in the sections that follow.
Chapter 2. Hardware components 23

2.2.1 Base features
There are two models of the BladeCenter JS20. Both models have the same
base features with the exception of the microprocessors used in the processor
subsystem. The JS20 Type 8842-21X Blade Server uses the PowerPC 970
microprocessor. The JS20 Type 8842-41X Blade Server uses the PowerPC
970FX microprocessor.
Processor subsystem
The processor subsystem of the BladeCenter JS20 is comprised of either two
PowerPC 970 or two PowerPC 970FX microprocessors. They operate at internal
clock frequencies of either 1.6 GHz or 2.2 GHz, respectively.
Each processor includes a 32 KB level 1 data cache, a 64 KB level 1 instruction
cache, and a 512 KB ECC level 2 cache.
Each processor is also interconnected with the rest of the system via a dedicated
interface to the memory controller and host I/O bridge (Northbridge). The
interface to each processor supports a maximum aggregate bandwidth of
6.4 GBps (for processors operating at 1.6 GHz) or 8.8 GBps (for processors
operating at 2.2 GHz).
You can learn more about the features and architecture of the processors in 2.3,
“PowerPC 970 and PowerPC 970FX Microprocessors” on page 29.
Memory subsystem
The memory subsystem is comprised of the memory controller and up to four
DDR1 333 MHz (PC2700) ECC memory modules. The memory controller uses
two-way interleaving to increase the available peak memory bandwidth to 5.3
GBps (333 MHz x 16 bytes).
The BladeCenter JS20 includes two 256 MB memory modules that provide a
total memory capacity of 512 MB. These memory modules are installed in slots 3
and 4. The remaining memory module slots (1 and 2) are empty in the base
BladeCenter JS20.
You can expand the available memory by installing optional memory modules.
The available memory modules are described in “Memory modules” on page 27.
I/O subsystem
The I/O subsystem is connected to the processor and memory subsystems via a
hyper transport link that supports an aggregate bandwidth of 1.6 GBps.
Downstream of this link is a range of I/O devices and expansion interfaces as
explained in the following sections.
24 The IBM Eserver BladeCenter JS20

Gigabit Ethernet controller
Each BladeCenter JS20 includes a Broadcom BCM5704s Gigabit Ethernet
controller, which provides the primary mechanism for connecting the
BladeCenter JS20 to other systems. This controller provides two independent
Gigabit Ethernet interfaces.
Each Gigabit Ethernet interface is connected to a separate BladeCenter I/O
module bay via the BladeCenter midplanes. The first interface is connected to I/O
module bay 1, and the second interface is connected to I/O module bay 2.
To use a Gigabit Ethernet interface, install a compatible I/O module in the
corresponding I/O module bay. Install a LAN Switch I/O Module in I/O module
bay 1 to support remote access to the BladeCenter JS20 text console via SoL.
I/O expansion option connectors
Each BladeCenter JS20 contains I/O expansion option connectors, which you
can use to attach one of several different expansion cards. This capability
enables you to expand the external I/O connectivity of the BladeCenter JS20 to
include Fibre Channel SANs, Myrinet high-bandwidth low-latency networks, and
additional Gigabit Ethernet LAN interfaces.
The expansion card installed in the I/O expansion option connectors can access
I/O modules installed in BladeCenter I/O module slots 3 and 4 via the
BladeCenter midplanes.
Note: The main I/O expansion option connector provides a PCI-X electrical
interface. However, a special form factor is required to enable installation
within the confined dimensions of a blade server. Therefore, standard PCI-X
adapter cards cannot be used with the BladeCenter JS20.
You can learn more about these expansion cards in “Expansion cards” on
page 28.
IDE controller
Each BladeCenter JS20 includes a dual-channel IDE controller and two
connectors for 2.5-inch IDE hard disk drives. You can use this capability to install
one or two optional 2.5-inch IDE hard disk drives on the BladeCenter JS20.
Note: You must install at least one hard disk drive for use as a boot device
unless you plan to install a Fibre Channel Expansion Card and configure the
BladeCenter JS20 to boot from a SAN. You can also boot from a network
interface to support the network installation of an operating system.
Chapter 2. Hardware components 25

You can learn more about the available hard disk drives in “Expansion cards” on
page 28.
USB controllers
Each BladeCenter JS20 includes two USB 1.1 host controllers. They are
connected via the BladeCenter midplanes to the USB peripherals located in the
media bay of the BladeCenter chassis.
You use these controllers when accessing either the CD-ROM or diskette drive
that are located in the media bay of the BladeCenter chassis. You can also
access other USB devices that may be attached to the USB connector located in
the media bay of the BladeCenter chassis.
Note: Verify that the USB device is supported by the operating system
installed on the blade server before you attach it to the USB connector located
in the media bay of the BladeCenter chassis.
You can assign only the USB peripherals to one blade server at a time. Either
use a button on the front of the blade server or one of the remote interfaces to the
Management Module to switch the USB peripherals to a specific blade server.
Miscellaneous I/O
Each BladeCenter JS20 incorporates various miscellaneous I/O devices that are
necessary to implement a functional server. These are illustrated in the
BladeCenter JS20 diagram in Figure 2-7 on page 23 and include:
► A real-time clock (RTC) for maintaining the date and time
► Non-volatile memory (NVRAM) for storing configuration settings
► Boot FLASH memory for storing the firmware (BIOS) for the blade server
► A serial port (UART)
The serial port is connected within the blade server to the BSMP. The BSMP
participates in the implementation of the SoL remote text console function that
provides remote connectivity to the serial port via the BladeCenter Management
Module.
Blade Systems Management Processor
Each BladeCenter JS20 has a BSMP. The BSMP provides the interface that
enables the BladeCenter Management Module to manage the BladeCenter JS20
via the BladeCenter chassis midplanes.
The BSMP is responsible for powering up the BladeCenter JS20, configuring the
various hardware subsystems, monitoring temperatures, and driving the status
indicators on the blade server front panel.
26 The IBM Eserver BladeCenter JS20

The BSMP also participates in the implementation of the SoL remote text
console function on the BladeCenter JS20.
2.2.2 Optional features
You can install several optional features on a BladeCenter JS20.
Memory modules
Each BladeCenter JS20 is equipped with 512 MB of memory. This is provided
using a pair of 256 MB memory modules.
You can also install the following optional memory modules on a BladeCenter
JS20:
► P/N 73P2275: IBM 512 MB 333 MHz PC2700 CL2.5 ECC DDR RDIMM
► P/N 73P2276: IBM 1 GB 333 MHz PC2700 CL2.5 ECC DDR RDIMM
You must install the optional memory module features in identical pairs. This is
necessary because the memory controller on the BladeCenter JS20 uses
two-way interleaving.
Two spare memory module slots (1 and 2) are available on the BladeCenter
JS20. You can expand the available memory to either 1.5 GB or 2.5 GB, using
either a pair of 512 MB or 1 GB memory modules respectively.
You can also remove the standard pair of 256 MB memory modules and replace
them by a pair of either 512 MB or 1 GB memory modules if further memory
expansion is required. To obtain the maximum supported memory configuration
of 4 GB, remove both of the base 256 MB memory modules and install four 1 GB
memory module features.
Hard disk drives
The base BladeCenter JS20 is not equipped with any hard disk drives. You can
install one or two of the 2.5-inch IDE hard disk drive option 40 GB 5400 rpm
ATA-100 Hard Disk Drive (P/N 48P7063).
When you install two hard disk drives, each hard disk drive is connected to its
own dedicated IDE bus.
Restriction: If you install an expansion card on a BladeCenter JS20, only one
hard disk drive can be installed on the same blade server.
You do not have to install a hard disk drive if you have installed a Fibre Channel
Expansion Card and have configured the BladeCenter JS20 to boot from a SAN.
Chapter 2. Hardware components 27

Expansion cards
You can install any one of the following expansion cards in the I/O expansion
option connectors of the BladeCenter JS20:
► P/N 73P9030: Gigabit Ethernet Expansion Card
► P/N 13N2203: Fibre Channel Expansion Card
► P/N 73P6000: Myrinet Cluster Expansion Card
We describe each of these expansion cards in the sections that follow.
Fibre Channel Expansion Card
You can use the Fibre Channel Expansion Card to connect the BladeCenter
JS20 to SANs. The Fibre Channel Expansion Card provides two 2 Gbps Fibre
Channel interfaces that are connected to I/O module bays 3 and 4. You must
install either a SAN switch I/O module or the Optical Pass-Thru I/O Module in one
or both of these I/O module bays to connect the blade server Fibre Channel
interfaces to an external SAN.
Note: There have been two different versions of the Fibre Channel Expansion
Card. Both versions are functionally equivalent. The earlier version (P/N
48P7061) is now withdrawn from marketing. However, you may still find it in
existing BladeCenter installations.
Gigabit Ethernet Expansion Card
You can use the Gigabit Ethernet Expansion Card to increase the number of
Gigabit Ethernet network interfaces on the BladeCenter JS20 from two to four.
The Gigabit Ethernet Expansion Card provides two Gigabit Ethernet interfaces
that are connected to I/O module bays 3 and 4. You must install either a LAN
Switch I/O Module or the Optical Pass-Thru I/O Module in one or both of these
I/O module bays to connect the Gigabit Ethernet interfaces on the expansion
card to an external LAN.
Myrinet Cluster Expansion Card
You can use the Myrinet Cluster Expansion Card to connect the BladeCenter
JS20 to a Myrinet network. Myrinet was developed by Myricom Inc., and
conforms to an ANSI standard (ANSI/VITA 26-1998).
Myrinet networks are typically used to support certain types of high performance
computing applications that distribute computation across a cluster of multiple
servers. Such applications are said to exploit distributed memory parallelism.
Distributed memory parallel applications often require the servers in the cluster
to exchange messages with each other through some form of interconnection
28 The IBM Eserver BladeCenter JS20

network. The performance of such applications often depends on how quickly
these messages can be exchanged. The highest performance is usually
obtained by using a high-bandwidth, low-latency interconnection network.
Myrinet provides a high-bandwidth, low-latency interconnection network that can
be used to optimize the performance of distributed memory parallel applications.
Consider using Myrinet if your application requires higher performance than is
achievable using Gigabit Ethernet technology.
You can learn more about Myrinet technology at the Myrinet Web site at:
http://www.myri.com
Note: The Myrinet Cluster Expansion Card is a repackaging of the Myricom
M3F-PCIXD-2 PCI-X adapter to fit the special form factor required for blade
server expansion cards.
When you install a Myrinet Cluster Expansion Card on a blade server, you must
install an Optical Pass-thru Module in I/O module bay 4 of the BladeCenter
chassis. You must also connect the external interfaces of the Optical Pass-thru
Module to an external Myrinet switch via appropriate optical fiber cables.
We elaborate on the planning implications associated with using the Myrinet
Cluster Expansion Card in 4.1.2, “High-performance, low-latency network
requirements” on page 58.
You can order Myrinet switches from IBM as a component of the IBM ^
Cluster 1350 product. You can learn more about Cluster 1350 at:
http://www.ibm.com/eserver/cluster
2.3 PowerPC 970 and PowerPC 970FX Microprocessors
This section provides more detailed information about the IBM PowerPC 970
Microprocessor and IBM PowerPC 970FX RISC Microprocessor that are used for
the main processors on the BladeCenter JS20.
This section reviews the POWER and PowerPC Architecture upon which the
PowerPC 970 and PowerPC 970FX microprocessors are based. It also
introduces the Vector/SIMD Multimedia eXtension (VMX) to the PowerPC
Architecture. And it describes the major features of the PowerPC 970 and
PowerPC 970FX microprocessors.
Chapter 2. Hardware components 29

2.3.1 Review of POWER and PowerPC Architecture
In 1990, IBM announced a new processor architecture called performance
optimization with enhanced RISC (POWER). This architecture was first
implemented in the IBM RS/6000®.
The POWER architecture was based on earlier IBM Research efforts to develop
reduced instruction set computer (RISC) technology. These efforts began in the
1970s and continued throughout the 1980s.
RISC and CISC concepts
In the 1960s and 1970s, most computers were of a design now known as a
complex instruction set computer (CISC). This type of design featured a large
and highly functional instruction set. Each instruction in a CISC processor design
typically needs several processor clock cycles to execute. An example of a CISC
design is the IBM System/360 and its successors.
The need for complex and highly functional instructions primarily existed
because computers were equipped with small quantities of slow memory.
Complex instructions resulted in fewer instructions per program, so less memory
was needed for program storage. Complex instructions were also easier for
human programmers to use when developing applications in assembly language
as single instructions could be rich in functionality.
However, studies showed that only a small percentage of CISC instructions were
commonly used by programs. Also, the benefit of the CISC approach diminished
over time as memory became larger and faster, and high-level languages
replaced assembly language.
The RISC concept was originally developed by IBM Fellow John Cocke in the
early 1970s and includes the following characteristics:
► A simple architecture with an optimized set of machine instructions
The instructions in a RISC architecture are typically simple in function and
regular in size. This simplifies the design of the instruction decode hardware.
The large number of infrequently used instructions typically found in CISC
architectures are eliminated and replaced by sequences of simple
instructions in RISC architectures.
► A high instruction execution rate
A common objective of early RISC processor designs was to execute an
average of one instruction per processor clock cycle. This has subsequently
been improved upon in superscalar RISC processor designs that can execute
multiple instructions per processor clock cycle.
30 The IBM Eserver BladeCenter JS20

► Optimizing compiler availability
RISC architectures assume the availability of good optimizing compilers, and
that most users will be developing applications in high-level languages. This
has simplified the hardware design by moving some of the complexity that
was previously in the hardware into the compiler. However, the simplified
instruction set of RISC architectures often makes it easier for compilers to
choose optimal instruction sequences.
► Load and store architecture
RISC architectures typically incorporate large numbers of processor registers,
and computational instructions manipulate values only in these registers.
Separate instructions are provided for loading data from memory and storing
data to memory. Compilers can take advantage of this clean separation and
schedule instructions in a way that keeps the processor busy despite the long
latency associated with memory access.
Today the difference between CISC and RISC is less clear. Many CISC
architectures have adopted many RISC concepts. Meanwhile, the instruction
sets of many RISC architectures have grown to include hundreds of instructions.
The original POWER and POWER2 processors
The first implementation of POWER architecture released by IBM was the
POWER processor as used in the first IBM RS/6000 servers and workstations.
This processor was a multiple-chip implementation of the POWER architecture
comprised of either seven or nine separate chips. It featured an 8 KB instruction
cache and either a 32 KB or 64 KB data cache. Over the life of the POWER
processor, it was shipped with clock speeds ranging from 20 MHz to 62.5 MHz.
The original POWER processor introduced the concept of a superscalar RISC
processor that could perform multiple operations in a single clock cycle by
exploiting multiple execution units that operated concurrently. At the time, this
represented a significant improvement over contemporary processor
implementations available from other manufacturers.
The floating-point performance of the POWER processor was further enhanced
by the presence of the fused multiply-add (FMA) instruction. It enabled two
floating-point operations to be executed by the floating-point execution unit each
processor clock cycle. The peak floating-point performance of the POWER
processor was therefore twice the processor clock rate. For example, a POWER
processor operating at 25 MHz had a peak floating-point performance of 50
MFLOPS.
In subsequent years, IBM introduced a single chip implementation of the
POWER processor called the RISC Single Chip (RSC), which was used in the
IBM RS/6000® Model 220 and Model 230.
Chapter 2. Hardware components 31

In 1993, IBM announced the POWER2 processor, which expanded upon the
POWER processor superscalar concept by adding additional execution units to
increase the number of operations that could be performed concurrently in a
single clock cycle. This processor was a multiple chip implementation of a
super-set of the original POWER architecture.
The POWER2 processor had two floating-point execution units, each able to
execute a FMA instruction every clock cycle. Therefore, the peak floating-point
performance of the POWER2 processor was four times the processor clock rate.
For example, a POWER2 processor operating at 55 MHz had a peak
floating-point performance of 220 MFLOPS.
The memory subsystem of the POWER2 was also enhanced in order to keep up
with the increased level of processor performance. This was achieved using
larger caches and wider paths to memory.
As a super-set of the POWER architecture, the POWER2 supported several
instructions that are not present on any other POWER or PowerPC Architecture
processors.
IBM subsequently delivered a single chip implementation of the POWER2 called
the POWER2 Super Chip (P2SC), which was offered at processor clock speeds
as high as 160 MHz, and delivered peak floating-point performance of 640
MFLOPS.
PowerPC Architecture
In 1991, IBM teamed with Apple Computer and Motorola to define the PowerPC
Architecture. The goals of this new architecture were to:
► Permit a broad range of implementations, from low-cost controllers to
high-performance processors
► Be sufficiently simple to permit the design of processors that have a very
short cycle time
► Minimize the effects that hinder the design of aggressive superscalar
implementations.
► Include multiprocessor features.
► Define a 64-bit architecture that is a super-set of the 32-bit POWER
architecture, providing application binary compatibility for 32-bit applications
While based on the POWER architecture, the PowerPC Architecture
incorporated several modifications to enable it to be more widely applied in a
variety of application scenarios. This vision has been subsequently realized with
processors implementing PowerPC Architecture now having been installed in
32 The IBM Eserver BladeCenter JS20

desktop, server, and embedded systems across commercial, consumer,
industrial, and scientific settings.
To achieve the design goals for the PowerPC Architecture, some features of the
original POWER architecture were removed. These were mostly features that
were infrequently used. None of the POWER2 extensions to the original POWER
architecture were included in the PowerPC Architecture.
The PowerPC Architecture defines both 32-bit and 64-bit modes of operation.
The primary differences in these two modes of operation are in the effective
length of addresses used by the processor, and the availability of extra
capabilities to manipulate double word (64-bit) fixed-point operands in 64-bit
mode. Floating-point capabilities are the same in both 32-bit and 64-bit modes.
The 32-bit PowerPC Architecture implementations only support the 32-bit mode
of operation. The 64-bit PowerPC Architecture implementations support both the
32-bit and 64-bit modes of operation. This enables the 64-bit PowerPC
Architecture implementations to support the full-speed execution of existing
32-bit applications, alongside 64-bit applications, in the same operating
environment.
The PowerPC Architecture is documented in a series of books. You can
download the most current version of the PowerPC Architecture books from the
IBM developerWorks® Web site at:
http://www.ibm.com/developerworks/eserver/articles/archguide.html
The architecture is divided into three parts, each one documented in a separate
book:
► Book I: PowerPC User Instruction Set Architecture
► Book II: PowerPC Virtual Environment Architecture
► Book III: PowerPC Operating Environment Architecture
Book IV is also available for each specific PowerPC processor implementation. It
describes extensions to the architecture that are specific to a particular
implementation.
We briefly review the PowerPC User Instruction Set Architecture, since this is the
part of the architecture that is of most interest to application programmers.
PowerPC User Instruction Set Architecture
The logical components of a PowerPC processor from the perspective of a
application program are:
► Fixed-point processor (FXU)
► Floating-point processor (FPU)
► Branch processor (BPU)
Chapter 2. Hardware components 33

These components all interact with storage as illustrated in Figure 2-8.
Instructions
from Storage
Fixed-Point
Instructions
Fixed-Point
Processing
(FXU)
Figure 2-8 PowerPC logical processing model
A PowerPC Architecture groups the instructions that an application program uses
on a PowerPC processor along the same lines into:
► Fixed-point instructions
► Floating-point instructions
► Branch instructions
All instructions in PowerPC Architecture are four bytes (one word) in size and are
aligned on word boundaries. This simplifies the decoding of instructions and is
characteristic of a RISC architecture.
The fixed-point instructions operate on a set of 32 general purpose registers
(GPRs), that are each 8 bytes (64 bits) in size on 64-bit PowerPC Architecture
processors. There is also a fixed-point exception register (XER). In 32-bit mode,
only the lower 32 bits of the GPRs are considered significant, and the double
word load and store instructions are not available.
The floating-point instructions operate on a set of 32 floating-point registers
(FPRs), that are each 8 bytes (64 bits) in size. There is also a floating-point
status and control register (FPSCR). The floating point capabilities are the same
in both 32-bit and 64-bit mode. The large number of GPRs and FPRs is also
typical of a RISC architecture.
34 The IBM Eserver BladeCenter JS20
Branch
Processing
(BPU)
Data to/from
Storage
Storage
Floating-Point
Instructions
Floating-
Point
Processing
(FPU)

The architecture also features a condition register that is set by the result of fixed
or floating point computational instructions and that can be used to control
branch processing. Branch instructions also make use of a count and a link
register.
Figure 2-9 illustrates the complete set of architected PowerPC registers used by
application programs.
CR
0 31
LR
0 63
CTR
0 63
GPR0
GPR1
...
GPR31
0 63
XER
0 63
FPR0
FPR1
...
FPR31
0 63
FPSCR
0 31
Condition Register
Link Register
Count Register
General Purpose Registers
Figure 2-9 PowerPC registers used by application programs
Fixed-Point Exception Register
Floating-Point Registers
Floating-Point Status
and Control Register
In the PowerPC Architecture, fixed and floating-point computational instructions
can operate on operands stored in registers. Data must be loaded from memory
into a register using a fixed or floating-point load instruction before computations
can be performed. Similarly, data must be stored from a register into memory
using an explicit fixed or floating-point store instruction once computation is
completed. This behavior is also characteristic of a RISC architecture.
Chapter 2. Hardware components 35

IBM processors implementing PowerPC Architecture
Since IBM teamed with Motorola and Apple Computer to define PowerPC
Architecture, IBM has developed many different processors that implement this
architecture. These processors fall into two main categories:
► Processors that are used as the main processor within IBM workstations and
servers
► Processors that IBM produces for embedding in a wide variety of computing
and non-computing devices
We focus our review of IBM processors implementing PowerPC Architecture on
those processors that are used as the main processors within IBM workstations
and servers.
32-bit PowerPC processors
In the early 1990s, IBM jointly designed with Motorola a series of 32-bit PowerPC
processors with the designation 60X that were used in IBM RS/6000
workstations and servers. They were also used in computer systems from other
computer manufacturers such as Apple Computer.
The first of these was the PowerPC 601®. It debuted in the IBM RS/6000 Model
250 in 1993. The PowerPC 601 was an evolution of the RISC Single Chip
implementation of the POWER processor and did not implement the full
PowerPC Architecture. It represented a transition between the earlier POWER
architecture and the PowerPC Architecture.
The PowerPC 601 was subsequently followed by the PowerPC 603 and
PowerPC 604. These represented full implementations of the 32-bit PowerPC
Architecture. Enhanced versions of these processors known as the PowerPC
603e and PowerPC 604e were subsequently introduced.
The PowerPC 603 was an entry-level processor. The PowerPC 604 was a
high-end processor and introduced support for symmetrical multi-processor
(SMP) configurations.
RS64 processors
In 1997, IBM introduced its first servers that incorporated processors based on
the 64-bit PowerPC Architecture. These processors were called the RS64. The
RS64 was subsequently enhanced with the introduction of the RS64-II, RS64-III,
and RS64-IV in later years. The RS64-IV operated at processor clock speeds as
high as 750 MHz.
The RS64 processor and its successors were optimized for use in commercial
applications and de-emphasized the floating point capabilities of the earlier
POWER2 processors. The RS64 processors were used in both IBM RS/6000
36 The IBM Eserver BladeCenter JS20

(later IBM Eserver pSeries®) and IBM AS/400® (later IBM Eserver iSeries)
servers, most frequently in SMP configurations, with some as large as 24
processors.
POWER3 and POWER3-II processors
The POWER3 processor was first used in IBM RS/6000 servers and
workstations introduced in 1998. The POWER3 processor continued the
evolution of the excellent floating-point performance provided by the POWER2
Super Chip (P2SC) processor, while implementing 64-bit PowerPC Architecture
and supporting SMP configurations of up to 16 processors.
Upon introduction, the POWER3 processor operated at 200 MHz, and delivered
peak floating-point performance of 800 MFLOPS per processor. An improved
version of the POWER3 processor, known as the POWER3-II, was subsequently
introduced. The POWER3-II featured processor clock speeds as high as 450
MHz, and delivered peak floating point performance of 1.8 GFLOPS per
processor.
You can learn more about the POWER3 processor in the IBM white paper
POWER3: Next Generation 64-bit PowerPC Processor Design, which is available
on the Web at:
http://www.ibm.com/servers/eserver/pseries/hardware/whitepapers/power3wp.html
POWER4 and POWER4+ processors
The POWER4 processor combined the best attributes of both the RS64 and
POWER3 processors. It delivers a 64-bit PowerPC Architecture processor that is
capable of delivering high levels of performance in both commercial and scientific
applications.
The POWER4 processor is the first PowerPC Architecture processor to feature
multiple processor cores on a single chip. Each POWER4 processor chip has
two independent processor cores and a shared level-2 cache.
The original POWER4 processor operated at processor clock speeds as high as
1.3 GHz and has featured in SMP configurations as large as 32 processor cores.
An enhanced version of the POWER4 processor, known as the POWER4+, was
subsequently introduced and installed at processor clock speeds as high as
1.9 GHz in the 32 processor core IBM Eserver pSeries Model 690.
You can learn more about the POWER4 processor in the IBM white paper
entitled POWER4 System Microarchitecture, which is available on the Web at:
http://www.ibm.com/servers/eserver/pseries/hardware/whitepapers/power4.html
Chapter 2. Hardware components 37

PowerPC 970 and PowerPC 970FX processors
These processors were first used by IBM in the BladeCenter JS20 that is the
subject of this book. We explore these processors in more detail in 2.3,
“PowerPC 970 and PowerPC 970FX Microprocessors” on page 29.
POWER5 processor
IBM recently introduced the POWER5 processor in the IBM Eserver i5 and p5
families of servers.
The POWER5 processor represents an evolution of the POWER4 processor.
Like the POWER4, the POWER5 processor implements two processor cores on
a single chip with a shared level-2 cache.
The major innovation in the POWER5 processor is the introduction of
symmetrical multi-threading (SMT) capabilities. SMT enables two threads of
execution (contexts) to execute simultaneously on the same processor core. This
can improve the performance of many applications through the exploitation of
otherwise idle execution units within the processor core.
The other significant innovation in the POWER5 processor design is the
integration of a memory controller onto the processor chip to decrease the
latency to memory and improve system reliability.
2.3.2 Vector/SIMD Multimedia eXtension
The Vector/SIMD Multimedia eXtension (VMX) is an extension to the PowerPC
Architecture. It defines additional registers and instructions to support
single-instruction multiple-data (SIMD) operations that accelerate data-intensive
tasks.
The VMX extensions to the PowerPC Architecture were developed jointly by
Apple Computer, IBM, and Motorola. Apple Computer and Motorola use different
terminology to refer to the VMX extensions of the PowerPC Architecture.
Specifically, Motorola uses the term Altivec, and Apple uses the term Velocity
Engine.
A short vector processing history
The basic concept behind vector processing is to enhance the performance of
data-intensive applications by providing hardware support for operations that can
manipulate an entire vector (or array) of data in a single operation. The number
of data elements operated upon at a time is called the vector length.
38 The IBM Eserver BladeCenter JS20

Scalar processors perform operations that manipulate single data elements such
as fixed-point or floating-point numbers. For example, scalar processors usually
have an instruction that adds two integers to produce a single-integer result.
Vector processors perform operations on multiple data elements arranged in
groups called vectors (or arrays). For example, a vector add operation to add two
vectors performs a pairwise addition of each element of one source vector with
the corresponding element of the other source vector. It places the result in the
corresponding element of the destination vector. Typically a single vector
operation on vectors of length n is equivalent to performing n scalar operations.
Figure 2-10 illustrates the difference between scalar and vector operations.
Scalar Add Operation
7
1
4 + 7 11
6
3
3
10
+
Figure 2-10 Scalar and vector operations
Vector Add Operation
Processor designers are continually looking for ways to improve application
performance. The addition of vector operations to a processor architecture is one
method that a processor designer can use to make it easier to improve the peak
performance of a processor. However, the actual performance improvements that
can be obtained for a specific application depend on how well the application can
exploit vector operations.
The concept of vector processing has existed since the 1950s. Early
implementations of vector processing (known as array processing) were installed
in the 1960s. They used special purpose peripherals attached to general
purpose computers. An example is the IBM 2938 Array Processor, which could
be attached to some models of the IBM System/360. This was followed by the
IBM 3838 Array Processor in later years.
By the mid-1970s, vector processing became an integral part of the main
processor in large supercomputers manufactured by companies such as Cray
Research. By the mid-1980s, vector processing became available as an optional
feature on large general-purpose computers such as the IBM 3090.
In the 1990s, developers of microprocessors used in desktop computers adapted
the concept of vector processing to enhance the capability of their
microprocessors when running desktop multimedia applications. These
5
6
11
4
5
2
12
7
17
7
8
12
Chapter 2. Hardware components 39

capabilities were usually referred to as Single Instruction Multiple Data (SIMD)
extensions and operated on short vectors. Examples of SIMD extensions in
widespread use today include:
► Intel Multimedia Extensions (MMX)
► Intel Streaming SIMD Extensions (SSE)
► AMD 3DNow!
► Motorola Altivec and IBM VMX
The SIMD extensions found in microprocessors used in desktop computers
operate on short vectors of length 2, 4, 8, or 16. This is in contrast to the classic
vector supercomputers that can often exploit long vectors of length 64 or more.
VMX extensions to PowerPC Architecture
The VMX extensions to PowerPC Architecture add a vector processor (VXU) to
the PowerPC logical processing model that was described earlier in “PowerPC
User Instruction Set Architecture” on page 33. This is illustrated in Figure 2-11.
Fixed-Point
Instructions
Fixed-Point
Processing
(FXU)
Instructions
from Storage
Figure 2-11 PowerPC with VMX logical processing model
The VXU operates on vectors that are a total of 128-bits long. These can be
interpreted by the VXU as either:
► A vector of sixteen 8-bit bytes
► A vector of eight 16-bit half words
► A vector of four 32-bit words
40 The IBM Eserver BladeCenter JS20
Branch
Processing
(BPU)
Floating-Point
Instructions
Floating-
Point
Processing
(FPU)
Storage
Data to/from
Storage
Vector
Processing
(VXU)
Vector
Instructions

The VMX extensions to PowerPC Architecture define 32 vector registers that
form the vector register file (VRF). The VRF is architecturally distinct from the
standard PowerPC FPRs and GPRs.
The VMX extensions to PowerPC also define two additional registers:
► The VMX status and control register (VSCR), which is used to control the
operation of the VXU and report the status of some VMX operations
► The VRSAVE register, which can be used to assist the operating system save
state across context switches by providing a mechanism for software to
indicate what vector registers are in use
The additional registers provided by VMX are illustrated in Figure 2-12.
VRSAVE
0 31
VSCR
0 31
Figure 2-12 VMX registers
VRSAVE Register
Vector Status and Control Register
Vector Register File (VRF)
VR0
VR1
VR31
0 127
The VMX extensions to PowerPC Architecture define new instructions that use
the VXU to manipulate vectors stored in the VRF. These instructions fall into
these categories:
► Vector integer arithmetic instructions (on 8-bit, 16-bit, or 32-bit integers)
► Vector floating-point arithmetic instructions (32-bit only)
► Vector load and store instructions
► Vector permutation and formatting instructions
► Processor control instructions used to read and write from the VMX status
and control register
► Memory control instructions used to manage caches
...
Chapter 2. Hardware components 41

For additional information about the topics presented in this chapter, we
recommend that you read the PowerPC Microprocessor Family: AltiVec
Technology Programming Environments Manual. You can find this manual on the
Web at:
http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/FBFA164F824370F987256
D6A006F424D
2.3.3 Features of the PowerPC 970 and PowerPC 970FX
The IBM PowerPC 970 Microprocessor is a recent implementation of 64-bit
PowerPC Architecture designed for use in workstations and small servers
requiring from one to four processors. The PowerPC 970 is based on the
processor core used in the POWER4 with the addition of a VXU to support the
VMX extensions to the PowerPC Architecture.
The IBM PowerPC 970FX RISC Microprocessor is an enhanced version of the
PowerPC 970 that takes advantage of improvements in semiconductor
manufacturing processes to deliver higher performance and lower power
consumption. In most respects, the PowerPC 970FX is identical to the PowerPC
970.
Table 2-3 lists the physical attributes of the PowerPC 970 and PowerPC 970FX.
Table 2-3 Physical attributes of PowerPC 970 and PowerPC 970FX
Physical attribute PowerPC 970 PowerPC 970FX
Process geometry 130 nm 90 nm
Physical size 125 mm 2
Unless otherwise noted, the following discussion applies to both the PowerPC
970 and PowerPC 970FX. References to PowerPC 970 also refer to the
PowerPC 970FX.
Figure 2-13 illustrates the internal structure of the PowerPC 970.
42 The IBM Eserver BladeCenter JS20
66 mm 2
Transistor count 58 million 58 million
Package pins 576 576

Figure 2-13 PowerPC 970 internal organization
Chapter 2. Hardware components 43

The major features of the PowerPC 970 include:
► Speculative, superscalar RISC processor core with vector extensions
► On-chip level-1 and level-2 caches
► Deeply pipelined design up to 25 stages long
► Aggressive branch prediction
► In order dispatch of up to five operations per cycle
► Out of order issue of up to ten operations per cycle
► In order completion of up to five operations per cycle
► Register renaming
► Up to 215 instructions in-flight
► Fast selective flush of incorrect speculative instructions and results
► Hardware prefetching of up to eight streams including four VMX streams
We elaborate on some of these features in the following sections.
Cache structure
The PowerPC 970 includes several on-chip caches to reduce memory latency
when fetching instructions and performing data load and store operations.
Understanding the structure of these caches is sometimes useful when doing
low-level assembly language programming.
The on-chip caches include:
► 64 KB, direct-mapped instruction cache
► 32 KB, 2-way set associative data cache
► 128-entry, instruction effective to real address translation (ERAT) cache
instructions
► 128-entry, data ERAT cache
► 64-entry, fully associative segment lookaside buffer (SLB)
► 1024-entry, 4-way set associative translation lookaside buffer (TLB)
► 512 KB, 8-way set associative, level-2 cache
Speculative superscalar features
The design of the PowerPC 970 uses a variety of techniques to enable
superscalar operation, where multiple instructions are executed during each
processor clock cycle. This capability is enabled by the exploitation of multiple
execution units within the processor core.
44 The IBM Eserver BladeCenter JS20

The speculative superscalar techniques used in the PowerPC 970 include:
► Aggressive branch prediction
– Prediction for up to two branches per cycle
– Support for up to 16 predicted branches in flight
– Prediction support for both branch direction and branch addresses
► In order dispatch of up to five operations into a distributed issue queue
structure per cycle
► Out of order issue of up to ten operations into ten execution pipelines per
cycle, comprised of:
– Two load or store operations
– Two fixed-point register-to-register operations
– Two floating-point operations
– One branch operation
– One condition register operation
– One VMX permute operation
– One VMX arithmetic operation
► Register renaming on most architected PowerPC registers including the
GPRs, FPRs, VRFs, condition register fields, FPSCR, VSCR, the link register
(LR), and the count register (CTR)
The total number of in-flight instructions within the processor core at any point in
time can be as high as 215.
The VMX execution units operate on multiple data elements concurrently.
Therefore, they provide a performance equivalent to multiple scalar execution
units.
Bus Interface Unit
The interface between the PowerPC 970 and external devices is provided by the
Bus Interface Unit (BIU). The interface is comprised of two unidirectional paths
(one in, one out) that are each four bytes wide.
The internal processor clock operates at an integral multiple of the interface
speed. On the BladeCenter JS20, the processor clock operates at double the
interface speed.
The total aggregate transfer rate across the interface in bytes per second is four
times of the processor clock speed (1/2 x 2 x 4). This equates to 6.4 GBps for
processors operating at 1.6 GHz, and 8.8 GBps for processors operating at
2.2 GHz.
The interface is multiplexed, with both addresses and data being transferred
across the same wires. Therefore the peak theoretical data throughput across
Chapter 2. Hardware components 45

the bus interface is slightly lower than the total aggregate bus transfer rate
derived above.
For more detailed information about the PowerPC 970 and PowerPC 970FX
microprocessors, refer to the IBM Microelectronics Division Web site at:
http://www.chips.ibm.com
To access the documentation, follow these steps:
1. Point your Internet browser to:
http://www.chips.ibm.com
2. Click Support.
3. Click Documentation.
4. Under Browse a product family, click PowerPC.
5. Under Categories, click PowerPC 9XX Microprocessors.
6. Under Categories, click PowerPC 970 and 970FX Microprocessors.
Various documents describe the PowerPC Architecture, programming, and 64-bit
computing.
46 The IBM Eserver BladeCenter JS20

Chapter 3. Software environment
3
This chapter discusses the software environments that are available for the IBM
Eserver BladeCenter JS20. It covers the operating systems and IBM
middleware supported on the BladeCenter JS20, as well as the system
management tools for the JS20.
© Copyright IBM Corp. 2005. All rights reserved. 47

3.1 Operating system support
3.1.1 AIX
The BladeCenter JS20 is supported by the AIX and Linux operating systems. At
the time of this writing, AIX 5.2H, AIX 5.3, Red Hat Enterprise Linux 3.0, and
SUSE LINUX Enterprise Server (SLES) are the supported operating systems on
this platform.
Both AIX and Linux operating systems work with a 64-bit kernel and mostly 32-bit
userland programs. Some of the included programs were compiled for 64 bit
when the code fully exploited the 64-bit address space.
AIX has evolved from its beginnings on the IBM RT to becoming the operating
system of choice for IBM’s largest UNIX servers. AIX is an enterprise operating
system that scales from workstations all the way up to massively parallel super
computers.
Certified as C2 security compliant by the US government, AIX also supports
industry-standard security features such as Pluggable Authentication Modules
(PAM), authenication by x.509 certificates, OpenSSH, Kerberos, and LDAP. AIX
fully exploits the features of the JS20 with its 64-bit kernel and support for over
16 TB of disk space.
With AIX installed, the JS20 can run thousands of software titles that were
written for the AIX platform, taking full advantage of AIX’s rich capabilities. At the
time of the writing of this book, AIX Version 5.2H and AIX 5.3 support the
BladeCenter JS20.
3.1.2 Red Hat Enterprise Linux
Red Hat has made its distribution of Linux available since 1994. Originally, Red
Hat offered its distribution for free download, and sold support contracts. In 2002,
Red Hat began marketing Red Hat Enterprise Linux (RHEL).
Unlike the original Red Hat distribution, RHEL was available only with a
maintenance contract from Red Hat. RHEL runs on x86_64, i386, ia64, ppc/64,
s390, and s390x platforms.
By early 2003, Red Hat stopped developing its non-enterprise distribution and
focused its efforts on RHEL. Red Hat elected to turn development of its free
software over to the community. This free community-supported distribution
became known as Fedora.
48 The IBM Eserver BladeCenter JS20

Red Hat Enterprise Linux 3 - Update 2 and later is supported on the JS20.
Although VMX exploitation was not supported at the time of this book’s
development in the version of GNU C Compiler (gcc) that ships with RHEL 3,
kernel support is available for programs that have been written to exploit VMX
instructions.
3.1.3 SUSE LINUX Enterprise Server
SuSE has been a Linux distributor since 1992 and was acquired by Novell in
early 2004. Originally, SuSE provided a free downloadable distribution, for which
support contracts could be purchased as desired. Eventually, their product line
was split. SuSE now markets SUSE LINUX Desktop, as well as SUSE LINUX
Enterprise Server.
SUSE LINUX Desktop is free to download, where SLES is available only with the
purchase of a maintenance contract. The desktop product is also available only
for x86_64 and IA32, where SLES supports IA32, ia64, ppc/64, s390, s390x, and
x86_64.
SLES Version 8 and Version 9 are supported on the BladeCenter JS20. SLES 9,
the latest version at the time of writing, ships with the 2.6 Linux kernel. This
brings significant gains to performance and scalability.
Both versions of SLES are VMX savvy and have versions of gcc that allow
manual exploitation of the vector engine.
3.2 System management tools
IBM has devised a strategy for systems management called the Universal
Manageability Initiative. There are several technologies that have been
incorporated under this umbrella. They all work toward the same goal: simple,
effective solutions for managing heterogeneous environments.
One of the main building blocks in this strategy is the use of simple yet powerful
software to oversee the management of many different systems. This section
investigates some of the possible options that are available for managing such
systems. It reviews the BladeCenter Web interfaces, IBM Director, Cluster
Systems Management, and xCAT.
3.2.1 BladeCenter Web interfaces
The BladeCenter Web interface allows system administrators to easily and
effectively manage up to 14 blades from an integrated interface. From trivial tasks
such and powering blades on or off, to more complex tasks such as firmware
Chapter 3. Software environment 49

3.2.2 IBM Director
management, the Web interface allows powerful control over all blades and
input/output (I/O) modules that are attached to the BladeCenter chassis.
Management of other BladeCenter resources, such as I/O modules, is also
possible from here, as well as the retrieval of system health information.
BladeCenter-specific features are also configured from here such as the Serial
over LAN (SoL).
IBM Director assists with the remote management of many IBM and non-IBM
machines including the BladeCenter JS20. The IBM Director console allows
system administrators to manage multiple BladeCenter chassis from a common
interface. It is the ideal solution for administering BladeCenter chassis in
heterogeneous environments or environments where a Director infrastructure
exists.
From the IBM Director console, system administrators can monitor resource
utilization. This key feature can be used for performance and capacity planning,
as well as to alert support staff of critical errors such as hardware failure.
IBM Director also allows the remote management of software. You can remotely
reconcile and install software from the console interface. This can be useful in
large environments for such resource-intensive tasks as patch management. By
using the Software Distribution Premium Edition, you can remotely install
patches to several servers at the same time with the click of a button.
You can learn more about IBM Director in 8.1, “IBM Director” on page 140.
3.2.3 IBM Cluster Systems Management
IBM Cluster Systems Management (CSM) provides many useful functions to
manage a cluster from a single point of control. These include resource
monitoring, automated monitoring and operation, remote hardware control,
remote command execution, security, configuration file management, parallel
network installation, and diagnostics.
By consolidating these capabilities, CSM helps to increase efficiency of an
administrator’s time and reduce their expenses. It helps administrators install
their clusters rapidly by automating many configuration tasks and by leveraging
existing open source products.
CSM also provides efficient monitoring of cluster resources without
overwhelming network bandwidth. The automated error detection CSM provides
helps catch problems before they impact the environment, and assists with rapid
50 The IBM Eserver BladeCenter JS20

esolution and recovery of problems that occur. CSM’s architecture and modular
construction maximizes flexibility so cluster solutions can evolve and grow as
needs change.
CSM is a collection of components that have been integrated to provide the basis
to construct and manage a cluster. Each of these components provides specific
capabilities related to the management of the cluster. This component-based
architecture provides flexibility for future expansion of the capabilities provided
by CSM. Each of the CSM components can be easily personalized to help meet
specific needs. For example, a cluster administrator can set up monitoring of
application processes and take actions if those processes disappear.
In short, CSM has been designed for large-scale cluster environments that
require simple control over multiple homonymous servers.
This book covers CSM in more detail in 8.2, “IBM Cluster Systems Management”
on page 147.
Chapter 3. Software environment 51

52 The IBM Eserver BladeCenter JS20

Chapter 4. Planning considerations
4
This chapter discusses planning considerations associated with the
implementation of the IBM Eserver BladeCenter JS20. Specifically it covers
network planning, operating system installation, and systems management.
© Copyright IBM Corp. 2005. All rights reserved. 53

4.1 Network planning
Successful installation of the BladeCenter JS20 requires that you have a clear
plan for how you will use the various networking capabilities of the BladeCenter
infrastructure. This plan should address the following questions:
► What network connectivity is needed for the blade servers to support the
applications installed on them?
► What network connectivity is needed to manage the BladeCenter,
input/output (I/O) modules, and blade servers?
► What Virtual local area networks (VLANs) are required for each LAN Switch
I/O Module to provide the network connectivity established previously?
► What IP subnet will be used for each VLAN and how will IP addresses be
allocated to each device on the VLAN?
► Will IP addresses be assigned statically or dynamically using DHCP?
► What host names will be used for each network interface?
► How will host names be resolved to IP addresses?
► Are there requirements for a high-performance, low-latency interconnection
network?
► Where multiple BladeCenter chassis are installed, how will they be
interconnected?
The following sections explore common network planning requirements that we
expect to arise in the installation of BladeCenter JS20s.
4.1.1 Minimal network requirements
At a minimum, most BladeCenter JS20 environments have the following network
requirements:
► A dedicated hardware management subnet: It is used to access both the
Management Module and management interfaces of I/O modules that are
installed in each BladeCenter chassis.
► A Serial over LAN (SoL) subnet internal to each BladeCenter chassis that
supports the SoL remote text console function: This is always implemented
using a LAN Switch I/O Module installed in I/O module bay 1.
► A subnet connected to each BladeCenter JS20: It is used to install and
manage the operating system on the blade server. Where you run AIX on
some blade servers, and Linux on other blade servers, you may need to use
multiple subnets for this purpose.
54 The IBM Eserver BladeCenter JS20

► One or more subnets connected to each BladeCenter JS20: They are used by
applications installed on the blade server to communicate other systems.
Figure 4-1 illustrates how these requirements can be provided in a logical
network view.
I/O
Modules
I/O
Module I/O
Module I/O
Module
JS20
Blade
Server
Application
Subnet
Figure 4-1 Network logical view
Hardware
Management
Subnet
JS20
Blade
Server
Management
Module
SOL
Subnet
Console Console Console Console Console Console
JS20
Blade
Server
JS20
Blade
Server
JS20
Blade
Server
OS
Management
Subnet
JS20
Blade
Server
eth0 eth1 eth0 eth1 eth0 eth1 eth0 eth1 eth0 eth1 eth0 eth1
The sections that follow describe each of the logical networks illustrated in
Figure 4-1.
Chapter 4. Planning considerations 55

Hardware management subnet
We recommend that you establish a dedicated hardware management subnet.
The 10/100BaseT Ethernet interface on the Management Modules installed in
each BladeCenter chassis provides the gateway to this subnet for each
BladeCenter chassis. You need to install external LAN switches or hubs to
interconnect the 10/100BaseT Ethernet interface on the Management Modules
of each BladeCenter chassis with external management systems.
You use this subnet to access the Management Module Web interface and
command line interface (CLI). You also use this subnet to access the Web
interface and CLI of I/O modules.
System management applications such as IBM Director or IBM Cluster Systems
Management (CSM) also use this subnet to communicate with the hardware
management functions of the BladeCenter infrastructure.
Restrict access to this subnet to those management systems, system
administrators, and operators who have responsibility for managing the
BladeCenter infrastructure.
You need to allocate multiple IP addresses to each BladeCenter chassis on this
subnet, including:
► One IP address for the external interface of the Management Module in each
BladeCenter chassis
► One IP address for the internal interface of the Management Module in each
BladeCenter chassis
► One IP address for the management interface of each I/O module in each
BladeCenter chassis
Note: Although the logical network view illustrated in Figure 4-1 shows the I/O
Management Module interfaces connecting directly to the hardware
management subnet, they are physically connected via the Management
Module, which acts as gateway to those interfaces. The Management Module
performs a proxy Address Resolution Protocol (ARP) function to make it
appear as though the I/O module management interfaces are attached to the
hardware management subnet.
It is possible to use a different subnet for the Management Module internal
network interface and I/O module management interfaces. However, we do not
recommend this configuration.
Learn how to configure these IP addresses in Chapter 5, “Hardware setup” on
page 69.
56 The IBM Eserver BladeCenter JS20

Serial over LAN subnet
The SoL remote text console function requires a subnet and underlying VLAN
that is implemented by a LAN Switch I/O Module installed in I/O module bay 1 of
the BladeCenter chassis (see Figure 2-2 on page 9). This subnet and VLAN is
entirely internal to each BladeCenter chassis and should not be externally
accessible.
If you use the 4-Port Gigabit Ethernet Switch Module or the Nortel Networks
Layer 2-7 Gigabit Ethernet Switch Module, the VLAN uses VLAN ID 4095. If you
use the Cisco Systems Intelligent Gigabit Ethernet Switch Module, you can
choose the VLAN ID.
You need to assign a unique range of IP addresses to this subnet for use by the
SoL remote text console function.
Important: One IP address is required for each blade server.
You need to specify only the starting IP address within the range of IP addresses
that you assign into the Management Module. The Management Module then
automatically assigns consecutive IP addresses from the starting address that
you provide to each blade server that you have installed.
You can learn how to configure the SoL subnet and VLAN in 5.5.1, “Configuring
Serial over LAN” on page 89.
Operating system management subnet
We expect most environments that use the BladeCenter JS20 to rely on the
network installation procedure to install the operating systems. We discuss this
further in 4.2, “Operating system installation” on page 60.
The operating system management subnet is used to support both the initial
installation and subsequent management of the operating systems installed on
BladeCenter JS20s. This subnet is implemented using a VLAN provided by the
LAN Switch I/O Modules installed in I/O module bay 2 of each BladeCenter
chassis. Alternatively you can implement it using a Pass-Thru I/O Module
installed in I/O module bay 2 that is connected to an external LAN switch.
You may want to install both AIX and Linux operating systems on different
BladeCenter JS20s in the same environment. In this case, you may need to set
up multiple operating system management subnets and underlying VLANs, one
for blade servers running AIX and the other for blade servers running Linux.
Chapter 4. Planning considerations 57

Application subnet
The primary reason you install BladeCenter JS20s is to support applications.
Many applications have requirements to communicate with other systems. You
use one or more application subnets for this purpose.
The primary application subnet is implemented using a VLAN provided by the
LAN Switch I/O Modules installed in I/O module bay 1 of each BladeCenter
chassis. The same LAN Switch I/O Module is used to support the SoL subnet
and VLAN.
Where different BladeCenter JS20s participate in different applications and there
is a requirement to separate application traffic, you may need to define multiple
application subnets and VLANs. Each BladeCenter JS20 is connected to the
appropriate application subnet based on the applications that are installed on the
blade server.
If application communication requirements with other systems are complex, you
can install an additional pair of Gigabit Ethernet interfaces on each BladeCenter
JS20. You do this using the Gigabit Ethernet Expansion Card in conjunction with
compatible I/O modules installed in I/O module bays 3 and 4.
4.1.2 High-performance, low-latency network requirements
Distributed memory parallel applications may require the installation of a
high-performance, low-latency interconnection network between BladeCenter
JS20s. This requirement can be supported through the use of an optional
Myrinet network.
To install a Myrinet network, you must install a Myrinet Expansion Card on each
BladeCenter JS20 that requires connectivity to the high-performance,
low-latency interconnection network. You must also install an Optical Pass-Thru
I/O Module in I/O module bay 4 of each BladeCenter chassis that contains blade
servers equipped with Myrinet Expansion Cards.
Then connect the Optical Pass-Thru I/O Module to external Myrinet switches to
complete the Myrinet network infrastructure. This is illustrated in Figure 4-2.
58 The IBM Eserver BladeCenter JS20

Blade
Server
Myrinet
Expansion
Card
Blade
Server
Myrinet
Expansion
Card
Break-out
Cables
Blade
Server
Myrinet
Expansion
Card
...
Optical Pass-Thru Module
I/O Module Bay 4
External
Myrinet
Switch
Figure 4-2 Myrinet network infrastructure
The Myrinet network infrastructure can be used to support application
programming interfaces such as the message passing interface (MPI) that are
often used by distributed memory parallel applications.
You can define IP addresses for each Myrinet network interface. You can also
use the Myrinet network infrastructure to support any application communication
based on IP protocols. For example, you can use this capability to support a
clustered file system such as IBM General Parallel File System (GPFS).
You can define a dedicated IP subnet for use with the Myrinet network
infrastructure that is distinct from the other IP subnets that are identified in 4.1.1,
“Minimal network requirements” on page 54.
4.1.3 Multiple BladeCenter environments
Blade
Server
Myrinet
Expansion
Card
Blade
Server
Myrinet
Expansion
Card
Blade
Server
Myrinet
Expansion
Card
Large configurations of BladeCenter JS20s require the installation of multiple
BladeCenter chassis. To support the network requirements identified previously,
you need to install external LAN switches to interconnect the BladeCenter
chassis with each other.
Chapter 4. Planning considerations 59

4.2 Operating system installation
There are two methods to install the JS20 in a BladeCenter. The first is an
attended method, using CD media. The second is a remote method through a
network installation. Using the CD installation media is similar to other pSeries
systems. You assign the CD-ROM to a blade, follow the installation steps
answering any prompts, and then repeat the process on the next blade.
Refer to the following Web site for instructions to install SuSE from CDs:
http://www.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm&lndocid=
MIGR-54819
With network installation, you can perform several installations at the same time.
The method is designed to reduce the installation time required when a large
number of blades requires operating system installation. This method was
chosen as the focus for the following sections.
4.2.1 Network installation planning
You can use two approaches to set up a network installation environment for the
BladeCenter JS20:
► Establish one or more network installation servers and manually initiate
network installation tasks.
► Use a systems management tool, such as IBM Cluster Systems Management
or xCAT, which can be used to automate much of the setup of network
installation servers and initiation of network installation tasks.
This book provides examples of both approaches. The remainder of this section
focuses on the planning considerations for the first approach. You can find
planning considerations for the second approach in 4.3, “Systems management”
on page 63.
Required network services
Network installation depends on several different network services:
► A bootstrap protocol (BOOTP) server or dynamic host configuration protocol
(DHCP) server
► A Trivial File Transfer Protocol (TFTP) server
► A Network File System (NFS) server
You can use one physical server to provide all three of these network services. If
you want to install AIX on BladeCenter JS20s, this server must run AIX. If you
want to install Linux on the BladeCenter JS20s, we recommend that you use the
60 The IBM Eserver BladeCenter JS20

same Linux distribution in the network installation server that you are planning to
install on the BladeCenter JS20s.
In some situations, you may want to install AIX on some BladeCenter JS20s and
Linux on other BladeCenter JS20s. While it is possible to use a single AIX server
to do this, we recommend that you establish two network install servers, one for
installing AIX and the other for installing your chosen Linux distribution.
Important: A potential problem may arise when you have multiple servers
acting as BOOTP or DHCP servers. Be careful when planning your network so
that they do not interfere with one another.
One approach is to place the BladeCenter JS20s running AIX and the AIX
network install server on a different VLAN from the BladeCenter JS20s
running Linux and the Linux install server. If you use this approach, you must
also disable the relay of BOOTP or DHCP requests in your network routers.
To learn how to set up AIX network install servers, see Chapter 7, “Installing AIX
on the JS20” on page 123. Or to set up Linux network install servers, see
Chapter 6, “Installing Linux” on page 101.
Setting up network installation
The BladeCenter JS20 firmware has the capability to boot an operating system
over a network using the BOOTP. You use this capability to initiate network
installation of the BladeCenter JS20.
The BOOTP protocol is a client-server protocol. When initiating a network
installation on a BladeCenter JS20, it behaves as a BOOTP client. Therefore,
you need to set up a BOOTP server to support initiating a network installation.
When you instruct the BladeCenter JS20 firmware to boot over a network using
BOOTP, it sends a request to the BOOTP server. The BOOTP server should
generate a response that contains the following information:
► The IP address, subnet mask, and default gateway that the BladeCenter JS20
should use for the network interface that sent the BOOTP request
► The IP address of a TFTP server that the BladeCenter JS20 should contact,
and the name of the file on that server that the BladeCenter JS20 should
request, to get the operating system installation boot image
The BOOTP protocol has been around since the mid1980s. In many
environments, BOOTP has now been replaced by the newer DHCP protocol.
DHCP was designed to interoperate with BOOTP, and most DHCP servers can
serve BOOTP clients.
Chapter 4. Planning considerations 61

When you perform network installations of AIX, you use the AIX BOOTP server.
When you perform network installations of Linux, you use a DHCP server that is
configured to support BOOTP clients.
When the BladeCenter JS20 firmware has received a response from a BOOTP
server, it contacts the TFTP server specified in the BOOTP response to load the
operating system’s installation boot image.
The operating system’s installation boot image then starts running on the
BladeCenter JS20. Eventually it contacts the Network File System (NFS) server
to obtain the files that it needs to perform the operating system installation.
Learn the details about the actual process of setting up network installation in
5.5.4, “Specifying IP parameters to Open Firmware” on page 96, and 7.1,
“Minimal NIM installation” on page 124.
Network interface selection
The BladeCenter JS20 has two standard Gigabit Ethernet interfaces. The first is
connected to I/O module bay 1, and the second is connected to I/O module bay
2. Refer to Figure 2-2 on page 9.
Restriction: The BladeCenter JS20 does not support the keyboard, video and
mouse (KVM) console supported by other blade servers. Instead the
BladeCenter JS20 only supports SoL remote text consoles via the
BladeCenter Management Module.
The SoL remote text console function uses the first Gigabit Ethernet interface on
each BladeCenter JS20 for communication between the Management Module
and the service processor found in each BladeCenter JS20. This interface is
known as eth0 under Linux and en0 under AIX.
Under normal conditions, using the Gigabit Ethernet interface by the SoL remote
text console function is entirely transparent. It does not impact usage of the
Ethernet interface by the operating system running on the BladeCenter JS20.
Unfortunately, the initiation of network installation using the BOOTP protocol from
the BladeCenter JS20 firmware temporarily disrupts the operation of the SoL
remote text console when it occurs on the same physical Gigabit Ethernet
interface. This disruption makes it difficult to diagnose problems with the network
installation process. Therefore, we recommend that you use the second Gigabit
Ethernet interface on each BladeCenter JS20 during network installation of the
operating system. The second interface is known as eth1 under Linux and en1
under AIX.
62 The IBM Eserver BladeCenter JS20

To use the second interface, you must have a LAN Switch I/O Module, or
Pass-Thru I/O Module connected to an external LAN switch, installed in I/O
module bay 2. You can learn more about why you should use the second Gigabit
Ethernet interface for network installation in the RETAIN® tip H181655 on the
Web at:
http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-55282
4.3 Systems management
4.3.1 IBM Director
The BladeCenter JS20 supports a rich set of systems management capabilities.
This section discusses planning considerations associated with exploiting some
of those systems management capabilities.
IBM Director is a systems management tool. It was originally developed to
provide a comprehensive management solution for servers and workstations
based on Intel microprocessors. Recently its capabilities have been expanded to
support management of other platforms such as the BladeCenter JS20. We
introduce IBM Director in 3.2.2, “IBM Director” on page 50.
This section reviews the major planning considerations associated with the
installation of IBM Director. For a comprehensive treatment of planning for IBM
Director, refer to the IBM Redbook Implementing System Management Solutions
using IBM Director, SG24-6188.
IBM Director has three components:
► IBM Director Server: This is the core of the IBM Director system
management solution.
► IBM Director Console: This provides the user interface that system
administrators and operators use to interact with IBM Director.
► IBM Director Agents: These provide the mechanism by which the IBM
Director can monitor and control a specific server and operating system
environment.
You can only install the IBM Director Agent on the BladeCenter JS20.
Chapter 4. Planning considerations 63

IBM Director Agent for the BladeCenter JS20
Support for the BladeCenter JS20 was introduced in IBM Director Version 4.12.
You can learn how to install the IBM Director Agent in 8.1.2, “Installing an IBM
Director Agent” on page 145.
We recommend that you install the IBM Director Server on an xSeries server.
For maximum functionality, this server needs IP connectivity to the Management
Module in each BladeCenter chassis and to every blade server. This connectivity
can be provided by connecting the IBM Director Server to both the hardware
management subnet and operating system management subnets described in
4.1.1, “Minimal network requirements” on page 54.
If your environment is primarily comprised of servers running Linux, consider
installation of the IBM Director Server under Linux.
4.3.2 IBM Cluster Systems Management
IBM Cluster Systems Management is a systems management tool. It simplifies
the management of clusters of servers running the AIX and Linux operating
systems.
This section focuses on the planning considerations that are associated with
using CSM to manage BladeCenter JS20s. For more planning information, refer
to the IBM Cluster Systems Management for Linux Planning and Installation
Guide, SA22-7873, and the IBM Cluster Systems Management for AIX5L
Planning and Installation Guide, SA22-7919.
Supported CSM and operating system releases
Preliminary support for the BladeCenter JS20 was introduced in CSM for Linux
Version 1.3.3.1.
CSM requires specific support for each operating system version that is running
on servers that it manages. Currently (at the time of publication) CSM Version 1.4
supports the following operating systems on the BladeCenter JS20:
► Red Hat Enterprise Linux Version 3 (QU3)
► AIX 5L Version 5.2 and 5.3
► SUSE LINUX Enterprise Server (SLES) Version 8.1and 9
To check for updates to this list of supported operating systems, refer to the CSM
publication site at:
http://publib.boulder.ibm.com/clresctr/windows/public/clusterbooks.html
64 The IBM Eserver BladeCenter JS20

Management node selection
Each CSM cluster requires a management node, which provides the central point
of administration and control for the entire cluster. CSM has complex rules
concerning the compatibility of management nodes, non-management nodes
(cluster nodes), and the operating system they run. We attempt to simplify these
rules by focusing exclusively on the requirements for a management node that
only manages BladeCenter JS20s. If your environment is more complex, refer to
the CSM planning documents.
You can use a CSM management node to both install and manage BladeCenter
JS20s, or you can use a management node to manage BladeCenter JS20s that
were installed using other mechanisms. Most environments use CSM to both
install and manage BladeCenter JS20s, so we focus on that scenario.
If you use a management node to both install and manage BladeCenter JS20s
running Linux, you must use the same type of Linux distribution on both the
management node and the BladeCenter JS20s. However, the management node
does not need the same processor architecture that is used in the BladeCenter
JS20.
For example, if you plan to install SLES on BladeCenter JS20s, you can use the
following types of management node and operating system combinations to both
install and manage the BladeCenter JS20s:
► A supported xSeries server running SLES
► A supported pSeries server running SLES
► A IBM HS20 Blade Server running SLES
► A IBM BladeCenter JS20 running SLES
In large installations, we recommend that you use either an xSeries or pSeries
server as a management node. If your environment also contains
non-BladeCenter JS20s, such as the HS20 Blade Server, or xSeries cluster
nodes, we recommend that you use an xSeries management node such as the
xSeries 345.
We performed our testing of CSM using an xSeries management node, since
many users of the BladeCenter JS20 may have environments that includes HS20
Blade Servers.
Chapter 4. Planning considerations 65

Network considerations and summary
IBM CSM is designed to work in the type of network environment that is outlined
in 4.1, “Network planning” on page 54, with multiple subnets that segregate
different types of network traffic. The CSM management node is normally
equipped with multiple Ethernet network interfaces that are connected as follows:
► One interface connects to the hardware management subnet to support
CSM’s hardware management functions.
► One interface connects to the operating system management subnet to
support CSM’s capabilities to install operating systems, maintain software
and configuration files, and provide ongoing monitoring of cluster nodes.
► One interface connects to an external network to enable remote access to the
management node by system administrators and operators.
For further information about the IBM Cluster Systems Management facility, refer
to the online documentation that is available.
66 The IBM Eserver BladeCenter JS20

Part 2 Implementing
Part 2
the
BladeCenter JS20
This part discusses implementation of the IBM Eserver BladeCenter JS20. It
covers planning considerations and setting up hardware. It explains how to install
AIX 5L and Linux. Plus it presents systems management scenarios.
© Copyright IBM Corp. 2005. All rights reserved. 67

68 The IBM Eserver BladeCenter JS20

Chapter 5. Hardware setup
5
This chapter explains the various activities that you must perform before you can
install an operating system on the BladeCenter JS20. Such activities include
configuring the Management Module and LAN switch modules and updating the
firmware. They also include configuring blade servers and the Serial over LAN
(SoL) remote text console function and using the Open Firmware interfaces.
© Copyright IBM Corp. 2005. All rights reserved. 69

5.1 Management Module configuration
This section describes the steps to set up the BladeCenter Management Module
so that you can work with BladeCenter JS20s. Advanced configuration of the
Management Module is beyond the scope of this book.
The primary setup task for the Management Module is to assign IP addresses,
which are necessary to communicate with the Management Module Web and
command line interfaces. To learn more about selecting these IP addresses, see
Chapter 4, “Planning considerations” on page 53.
The Management Module has two network interfaces, an external interface
(eth0) and an internal interface (eth1). The external interface is accessible via the
10/100BaseT connector on the Management Module. The internal interface is
connected to the management interfaces of all the installed I/O modules that
support such interfaces. This includes all switch I/O modules.
The default behavior of a new Management Module is to request an IP address
for the external interface via DHCP. If the Management Module does not receive
a valid response from a DHCP server within two minutes of powering on, it uses
a default static IP address of 192.168.70.125 with the default subnet mask of
255.255.255.0.
Note: The default host name is MMxxxxxxxxxxxx, where xxxxxxxxxxxx is the
burned-in medium access control (MAC) address.
The default IP address for the internal interface is statically assigned to be
192.168.70.126 with a subnet mask of 255.255.255.0.
You can reset the IP addresses of a Management Module that was previously
configured back to the factory defaults by using the IP reset button shown in
Figure 5-1 on the Management Module. You can find the procedure for doing this
in the IBM Eserver BladeCenter Management Module User’s Guide.
70 The IBM Eserver BladeCenter JS20

Figure 5-1 Management Module controls and indicators
Configuring the Management Module
Use the following procedure to set the IP addresses for the Management Module
external and internal interfaces:
1. Connect the Management Module to an isolated private Ethernet network.
Also connect a workstation that has a Web browser to the same isolated
private Ethernet network.
2. Configure a static IP address for the workstation Ethernet interface that is in
the same subnet as the Management Module default IP addresses.
For example, we used a static address of 192.168.70.100 with a subnet mask
of 255.255.255.0 for the workstation Ethernet interface. Do not use addresses
in the range of 192.168.70.125 through 192.168.70.130. These addresses
conflict with the default addresses assigned by the Management Module.
3. Connect to the Management Module Web interface by pointing your Web
browser on the workstation to:
http://192.168.70.125
4. Enter a valid user ID and password. The factory default configuration of a
Management Module defines a user ID named USERID with a password of
Chapter 5. Hardware setup 71

PASSW0RD. The 0 in the password is a zero. In production environments,
consider changing these defaults.
5. From the Management Module Web interface, select MM Control → Network
Interfaces.
6. The form window shown in Figure 5-2 is displayed. Enter the desired external
and internal IP addresses, subnet masks, and default gateway for the
Management Module. Then click Save to store the new IP addresses.
Figure 5-2 IP address configuration
72 The IBM Eserver BladeCenter JS20
Note: The default user ID is USERID, and the default password is
PASSW0RD. The number zero is between the W and the R.

7. Restart the Management Module. In the Management Module Web interface,
select MM Control → Restart MM. Then click the Restart button on the
displayed form. You are prompted to confirm the restart before it occurs.
8. Remove the Management Module from the isolated private Ethernet network
and connect it to the network you will use to manage the BladeCenter.
You can now connect to the Management Module Web and command line
interfaces using the IP address that you assigned to the Management Module
external network interface.
Now consider performing other Management Module setup tasks, such as:
► Setting the Management Module date and time so that log entries have useful
timestamps
► Defining user IDs and passwords for the system administrators and operators
who will manage the BladeCenter
Alternatively, you can configure the Management Module to use a Lightweight
Directory Access Protocol (LDAP) directory for this purpose.
► Configuring the Management Module to send alerts to management systems
via SNMP, or system administrators using e-mail via Simple Mail Transfer
Protocol (SMTP)
► Enabling the use of Secure Sockets Layer (SSL) to securely access the
Management Module Web interface
► Enabling the use of Secure Shell (SSH) to securely access the Management
Module command line interface (CLI)
For additional information about how to perform these tasks, refer to the
IBM Eserver BladeCenter Management Module User’s Guide.
5.2 LAN Switch I/O Module configuration
This section explains the basic set up LAN Switch I/O Modules so you can work
with BladeCenter JS20s. Advanced configuration of LAN Switch I/O Modules is
beyond the scope of this book.
The primary setup tasks for LAN Switch I/O Modules are to:
► Assign IP addresses to the LAN Switch I/O Module management interfaces
► Enable the LAN Switch I/O Module external Ethernet ports
You can learn about the selection of IP addresses for the LAN Switch I/O Module
management interfaces in Chapter 4, “Planning considerations” on page 53.
Chapter 5. Hardware setup 73

When a new LAN Switch I/O Module is first installed, the Management Module
assigns a default IP address to the management interface of the LAN Switch I/O
Module. The default IP address is chosen based on the I/O module bay where
the LAN Switch I/O Module is installed. Those I/O modules installed in I/O
module bays 1, 2, 3, and 4 are assigned IP addresses 192.168.70.127,
192.168.70.128, 192.168.70.129, and 192.168.70.130, respectively.
Set the IP address of each LAN Switch I/O Module management interface and
enable the external ports:
1. Connect to the Management Module using the Web browser interface as
explained in “Configuring the Management Module” on page 71.
2. From the Management Module Web interface, select I/O Module Tasks →
Management.
3. In the form shown in Figure 5-3, scroll down to the entry for the LAN Switch
I/O Module that you want to configure. Enter the IP address, subnet mask,
and gateway that you want to assign to the LAN Switch I/O Module
management interface. Click the Save button to activate the new IP address.
Figure 5-3 LAN Switch I/O Module IP address
74 The IBM Eserver BladeCenter JS20

4. You are prompted to confirm that you want to change the IP address.
5. Select the Advanced Management link for the LAN Switch I/O Module.
6. Scroll down to the Advanced Setup section of the displayed form. This is
illustrated in Figure 5-4. For the External Ports field, select Enabled from the
drop-down list and then click the Save button.
Figure 5-4 LAN Switch I/O Module setup
Chapter 5. Hardware setup 75

At this point, the LAN Switch I/O Module management interface has an IP
address. Also, the external ports on the LAN switch module are enabled so that
they can be used to communicate with blade servers.
The SoL remote text console function of the Management Module depends on a
VLAN provided by a LAN Switch I/O Module installed in I/O module bay 1. This
VLAN is automatically provided by the 4-Port Gigabit Ethernet Switch Module
and the Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module using VLAN
4095.
If you are using a Cisco Systems Intelligent Gigabit Ethernet Switch Module in
I/O module bay 1, you must manually configure this VLAN. A procedure is
documented in the Cisco Systems Intelligent Gigabit Ethernet Switch Module for
the IBM Eserver BladeCenter Installation Guide. This Cisco guide describes
how to manually set up the VLAN that is needed to support the SoL remote text
console function.
5.3 Blade server configuration
Minimal configuration is required for each BladeCenter JS20 prior to installing an
operating system. The two main configuration tasks are:
► Assigning a name to the blade server
► Setting the blade server boot sequence
The easiest way to perform these tasks is through the Management Module Web
interface.
76 The IBM Eserver BladeCenter JS20

5.3.1 Assigning names to blade servers
Set the name of each blade server in a BladeCenter chassis by using these
steps:
1. From the Management Module Web interface, select Blade Tasks →
Configuration.
2. In the Blade Information section of the form, as shown in Figure 5-5, type the
name that you want to assign to each blade server.
3. Scroll down the form and select the Save button to save the names that you
have assigned to each blade server.
Figure 5-5 Blade server naming
Chapter 5. Hardware setup 77

5.3.2 Setting the boot sequence
Set the boot sequence of each blade server by using this procedure:
1. From the Management Module Web interface, select Blade Tasks →
Configuration.
2. Scroll down the form to the Boot Sequence section to see the current boot
sequence for all the blade servers.
3. If a blade server does not have the correct boot sequence, you can change
the boot sequence by selecting the blade server name, which displays the
form shown in Figure 5-6.
4. Set the desired boot sequence using the drop-down lists and click the Save
button to set the new boot sequence.
The correct boot sequence depends on the method that you plan to use to install
an operating system on the blade server.
Figure 5-6 Boot sequence of blades
78 The IBM Eserver BladeCenter JS20

5.4 Firmware
You may need to upgrade various firmware components before you install an
operating system on a BladeCenter JS20. The minimum firmware levels required
and the actual firmware levels we used during the development of this book are
listed in Table 5-1.
To check the level of firmware components, use the following two steps:
1. Find the latest firmware levels and update packages distributed on the IBM
support Web site.
a. Go to the IBM support Web site:
http://www.ibm.com/pc/support
b. Under Download, select Driver matrices.
c. On the Personal computing drivers Web page, select Servers.
d. On the Software and device drivers - Servers Web page, select eServer
BladeCenter.
e. On the “Software and device drivers - IBM eServer BladeCenter (Type
8677)” Web page, you see the firmware levels for the BladeCenter.
f. You can also select IBM eServer BladeCenter JS20 - Software and
device drivers to see JS20-specific firmware levels.
2. Identify the current build levels and levels of firmware that are currently
installed by displaying the firmware vital product data (VPD). You can view the
firmware VPD for most firmware components, as shown in Table 5-1, through
the Management Module Web interface. This is illustrated in Figure 5-7.
Table 5-1 BladeCenter and BladeCenter JS20 firmware components
Firmware component Minimum level Level we used
Management Module V1.09 (BRET58A) V1.14 (BRET67I)
4-Port Gigabit Ethernet
Switch Module
V1.04 (ibmrun.081) V1.07 (ibmrun.095)
BladeCenter JS20 BIOS FW04310120 FW04310120
BladeCenter JS20 BSMP V1.00 (BQ8T16A) V1.0 (BQ8T20A)
Note: FW04310120 for the JS20 BIOS is the minimum required supported
level to support AIX 5L releases like V5.2 (ML4).
Chapter 5. Hardware setup 79

Figure 5-7 Viewing firmware levels on BladeCenter
The following sections explain how to update the various firmware components
associated with the BladeCenter JS20.
5.4.1 Management Module firmware
The easiest way to upgrade the firmware of the Management Module is through
the Management Module Web interface.
1. Download the firmware update package from the IBM support Web site. This
is usually a ZIP file that contains several files with a PKT extension.
80 The IBM Eserver BladeCenter JS20

2. Unzip the firmware update package into a directory on the workstation where
you are running the Web browser that you are using to connect the
Management Module Web interface.
3. Update each PKT file to the Management Module as explained in the
following steps.
a. In the navigation panel of the Management Module Web interface
(Figure 5-8), select MM Control → Firmware Update.
b. In the Update MM Firmware pane (right side of Figure 5-8), select the
Browse button and locate the PKT file in the directory where you
unpacked it.
c. Click the Update button to initiate the download of the PKT file to the
Management Module. This step may take some time to complete.
Figure 5-8 Updating the Management Module firmware
d. A window opens that shows you the progress of the download.
e. Click the Continue button when prompted to confirm the firmware update.
This step may also take some time to complete.
f. Another window opens that shows you the progress.
Chapter 5. Hardware setup 81

4. After you download all the PKT files to the Management Module, restart the
Management Module to make the firmware updates active. In the
Management Module Web interface, select MM Control → Restart MM.
5.4.2 LAN Switch I/O Module firmware
The method you should use to upgrade the firmware of a LAN switch module
depends on the type of switch module. We demonstrate how you can upgrade
firmware for the 4-port Gigabit Ethernet Switch Module, which is the switch
module we used in our testing.
For information about upgrading firmware in the Nortel Networks Layer 2-7
Gigabit Ethernet Switch Module, refer to the Nortel Networks Layer 2-7 GbE
Switch Module for IBM Eserver BladeCenter Installation Guide.
For information about upgrading firmware in the Cisco Systems Intelligent
Gigabit Ethernet Switch Module, refer to the Cisco Systems Intelligent Gigabit
Ethernet Switch Module for the IBM Eserver BladeCenter Installation Guide.
1. Download the firmware update package from the IBM support Web site. This
is usually a ZIP file that contains the firmware image file.
2. Unzip the firmware update package into a directory on the workstation where
you are running the Web browser that you use to connect the Management
Module Web interface.
3. Before you can upgrade the firmware of the 4-Port Gigabit Ethernet Switch
Module, configure an IP address for the switch module management
interface. This enables the capability to directly connect to the switch module
Web interface. To configure the IP address of the switch module management
interface, see 5.2, “LAN Switch I/O Module configuration” on page 73.
4. Connect to the switch module Web interface via the Management Module
Web interface using the following procedure:
a. In the Management Module Web interface, select I/O Module Tasks →
Management.
b. Select the Advanced Management link for the I/O module bay where the
switch module is installed.
c. Scroll down to the bottom of the page to the Start Telnet/Web Session
category. Click the Start Web Session button.
d. A new window opens for the switch module Web interface. You are
prompted to provide a user ID and a password. The default user ID is
USERID, and the default password is PASSW0RD, where the number 0 is
between the letters W and R in the password. Consider changing the
defaults in a production environment.
82 The IBM Eserver BladeCenter JS20

You can also connect directly to the switch module user interface from any
Web browser if you know the IP address of the switch module.
5. Upgrade the firmware using the following procedure:
a. From the switch module Web interface, select Maintenance → Using
Browser → Upgrade Firmware/Configuration File.
b. You now see the form shown in Figure 5-9. Select the Browse button and
locate the switch module firmware file in the directory where you put it
earlier.
c. Select the Start button to initiate the switch module firmware update
process.
d. A window opens requesting confirmation. After you confirm the update,
the main window displays the status of the update until it completes and
reboots the switch module.
Figure 5-9 Firmware upgrade of 4-Port Gb Ethernet Switch Module
6. After you restart the switch module, verify that the IP address of the switch
module is still set correctly through the Management Module Web interface. In
some cases, you may need to reset the IP address.
Chapter 5. Hardware setup 83

5.4.3 BladeCenter JS20 firmware (BIOS)
There is currently no way to upgrade the firmware (BIOS) of a BladeCenter JS20
without first installing an operating system on the blade server. This is usually not
an issue, because newly shipped BladeCenter JS20s already have current
firmware installed.
For completeness, we explain how to upgrade the firmware of a BladeCenter
JS20 after you install an operating system.
Upgrading firmware under AIX
AIX includes the update_flash utility as part of the AIX diagnostics package. To
use the update_flash utility, you must first download the latest firmware from the
IBM support Web site. Then transfer it to a disk that is accessible to the operating
system running on the BladeCenter JS20. Next, you start the update_flash utility
from the directory where the new firmware is located. The update_flash utility
requires that you have root privileges.
The following example illustrates the usage of update_flash:
/usr/lpp/diagnostics/bin/update_flash -f JS1FW419A.IMG
In this example, JS1FW419A.IMG is the firmware image that was previously
downloaded from the IBM support Web site. The update_flash utility asks you to
confirm that you want to update the firmware and then reboot the operating
system to perform the actual update.
If you subsequently have a problem with the new firmware and want to revert to
the previous firmware level, use the following command:
/usr/lpp/diagnostics/bin/update_flash -r
Alternatively, if you are satisfied with the new firmware, commit the firmware
update before you install any future firmware by using this command:
/usr/lpp/diagnostics/bin/update_flash -c
Upgrading firmware under Linux
The mechanism used to update the firmware (BIOS) of the BladeCenter JS20
under the Linux operating system is based on a Linux kernel module called
rtas_flash. This kernel module is available in most recent 2.4 kernels and in all
2.6 kernels. All supported Linux distributions for the BladeCenter JS20 include
this kernel module.
The rtas_flash kernel module creates interfaces for manipulating the flash
memory where the firmware is stored in the /proc/ppc64/rtas directory. In most
Linux distributions, this kernel module is normally not loaded at system boot.
84 The IBM Eserver BladeCenter JS20

Therefore, the interfaces to manipulate the flash memory are not normally
present in /proc/ppc64/rtas.
IBM provides a script for Linux called update_flash that uses the interfaces
provided by the rtas_flash kernel module. It also supports the same options and
syntax as the update_flash utility under AIX.
You can obtain the latest version of the update_flash script by installing the
diagnostic utilities for Linux on POWER that IBM distributes from the Web at:
http://techsupport.services.ibm.com/server/lopdiags
Note: Some Linux distributions provide a version of the update_flash script
that is not fully functional. Always obtain the latest version of the update_flash
script from IBM before you attempt to update firmware.
To use the update_flash script, follow these steps:
1. Download the latest firmware from the IBM support Web site.
2. Transfer it to a disk that is accessible to the operating system running on the
BladeCenter JS20.
3. Enter the update_flash command from the directory where the new firmware
is located. The update_flash command requires that you have root privileges.
The following example illustrates the usage of update_flash:
/usr/sbin/update_flash -f JS1FW419A.IMG
In this example, JS1FW419A.IMG is the firmware image that was previously
downloaded from the IBM support Web site. The update_flash utility copies the
new firmware image into the kernel and then reboots the operating system to
perform the actual firmware update.
If you subsequently have a problem with the new firmware and want to revert to
the previous firmware level, then use the following command:
/usr/sbin/update_flash -r
If you are happy with the new firmware, commit the firmware upgrade before you
install any future firmware by using the following command:
/usr/sbin/update_flash -c
5.4.4 Integrated Systems Management Processor firmware
The easiest way to upgrade the firmware of the BladeCenter JS20 Integrated
Systems Management Processor (ISMP) is through the Management Module
Web interface.
Chapter 5. Hardware setup 85

1. Download the firmware update package from the IBM support Web site. This
package contains a file with a ZIP extension, which is the firmware for the
ISMP.
2. Place the file into a directory on the workstation where you will run the Web
browser that you will use to connect the Management Module Web interface.
3. Download the PKT file to the ISMP. Follow this upgrade procedure:
a. As shown in Figure 5-10, select Blade Tasks → Firmware Update in the
Management Module Web interface.
b. The Update Blade Firmware pane is displayed (right side of Figure 5-10).
In the Target drop-down list, select the blade server on which you want to
upgrade the ISMPfirmware.
c. Click the Browse button and locate the ISMP firmware ZIP file in the
directory where you placed it earlier.
d. Click the Update button to initiate download of the ZIP file to the
Management Module. This step may take some time to complete.
Figure 5-10 Blade firmware update
86 The IBM Eserver BladeCenter JS20

e. A window opens that shows the progress of the download.
f. Click the Continue button when prompted to confirm the firmware update.
This step may also take some time to complete.
g. A window opens that shows the progress.
The new ISMP firmware is now active in the target BladeCenter JS20.
5.5 Providing a console for the JS20 blades
The BladeCenter JS20 differs from other blade servers that are available for the
BladeCenter in that it does not provide an interface to a keyboard, video monitor,
and mouse (KVM) console. Therefore, you must set up the SoL remote text
console function to provide a console for a BladeCenter JS20. Use of SoL is
optional for other types of blade server that support KVM consoles.
The SoL remote text console function involves several different components in
the BladeCenter infrastructure as illustrated in Figure 5-11.
The SoL remote console function works as follows:
1. Using a Telnet or SSH client, connect to the BladeCenter Management
Module CLI. This is usually via an external management network that is
connected to the Management Module’s 10/100BaseT Ethernet interface.
2. From the Management Module’s CLI, initiate a SoL remote console session to
the desired blade server.
3. The Management Module uses a private VLAN provided by the LAN switch
module in I/O module bay 1, to transport the SoL data stream to the Ethernet
interface of the target blade server.
4. The Ethernet controller of the target blade server passes the SoL data stream
received from the private VLAN to the blade system management processor
(BSMP), which manages the text console for the blade server.
Chapter 5. Hardware setup 87

LAN Switch
Module in
I/O Module
Bay 1
BSMP
Figure 5-11 Serial over LAN components
The sections that follow explain how to configure and use the SoL remote text
console function.
88 The IBM Eserver BladeCenter JS20
Telnet
or
SSH Client
External Management Network
Ethernet
Interface
(eth0)
Management
Module
External Interface
(eth0)
Internal Interface
(eth1)
Ethernet
Interface
(eth1)
Ethernet Controller
JS20 Blade Server
I/O Module
Bay 2

5.5.1 Configuring Serial over LAN
Before you attempt to configure the SoL remote text console function, verify that
you have all the prerequisites in place:
► A supported LAN switch module installed in I/O module bay 1
This switch module is used to provide a VLAN that connects the Management
Module to the first Ethernet interface on each blade server.
► The minimum firmware levels described in 5.4, “Firmware” on page 79
This is important if you install a BladeCenter JS20 in an existing BladeCenter
chassis that may have back-level firmware in the Management Module or LAN
switch modules.
► A reliable Telnet or SSH client
► A network connecting the Telnet or SSH client to the BladeCenter
Management Module external 10/100BaseT Ethernet interface
► An identified range of IP addresses that will be used by the Management
Module to communicate with the BSMP on each blade server via the private
VLAN
This was discussed in Chapter 4, “Planning considerations” on page 53.
The easiest way to configure the SoL remote text console function is through the
Management Module Web interface as explained in the following steps.
To configure the SoL remote text console function, use the following procedure:
1. From the Management Module Web interface, select Blade Tasks → Serial
Over LAN.
2. In the right pane, scroll down until you see the Serial Over LAN Configuration
section (see Figure 5-12). Complete the following tasks:
a. From the Serial over LAN list, select the Enabled option.
b. Leave the value for SoL VLAN ID at the default (4095) if you have either a
4-port Gigabit Ethernet Switch Module or a Nortel Networks Layer 2-7
Gigabit Ethernet Switch Module installed in I/O module bay 1.
If you have a Cisco Systems Intelligent Gigabit Ethernet Switch Module
installed in I/O module bay 1, set the VLAN ID to the same value you used
when you manually defined the VLAN that supports SoL.
c. Enter the start of the IP address range that will be used by the
Management Module to communicate with the BSMP on each blade
server.
d. Leave the values for Accumulate timeout, Send threshold, Retry count,
and Retry interval at their defaults (5, 250, 3, and 250).
Chapter 5. Hardware setup 89

e. Leave the values for User Defined Keystroke Sequences at their defaults.
f. Click the Save button.
Figure 5-12 Serial over LAN configuration
3. Restart the Management Module.
a. In the Management Module Web interface, select MM Control → Restart
MM.
b. Click the Restart button on the displayed form.
c. You are prompted to confirm the restart before it occurs.
4. After the Management Module has restarted and you have reconnected to the
Management Module Web interface from your Web browser, enable the SoL
remote text console for each blade server.
a. Select Blade Tasks → Serial Over LAN from the Management Module
Web interface.
b. In the right pane, scroll down until you see the Serial Over LAN Status
section (see Figure 5-13).
c. Select the blade servers you want to enable for SoL. You can choose all of
them by clicking the check box at the top of the table.
90 The IBM Eserver BladeCenter JS20

d. Click the Enable Serial Over LAN link underneath the table. You may
need to scroll down to see this link.
Figure 5-13 Serial over LAN status
e. After a few seconds, the window refreshes. In the SoL column of the table,
verify that each blade server has a status of Enable.
The configuration of the SoL remote text console function is now complete.
5.5.2 Using Serial over LAN
Access to the SoL remote text console function is via the Management Module
CLI. The CLI is documented in the IBM ^ BladeCenter Management
Module Command-Line Interface Reference Guide.
You can connect to the Management Module CLI using either a Telnet or SSH
client. Figure 5-14 shows an active Telnet session that displays the help
command. The Management Module can support up to 20 concurrent CLI
Chapter 5. Hardware setup 91

connections. This is sufficient to support the concurrent use of a SoL remote text
console to each blade server in a full BladeCenter chassis. At the same time, it
supports six additional CLI connections for other administrative activities.
Figure 5-14 Management Module’s command line interface
Restriction: You can only have one active SoL remote text console
connection to each blade server.
The Management Module CLI is context sensitive. When you issue CLI
commands, you can accept the current context or override the context using the
-T option provided on most commands. In our examples, we use the -T option to
make it clear on which entity the command is operating. You can change the
context for the Management Module CLI by using the env -T command.
92 The IBM Eserver BladeCenter JS20

To use the SoL remote text console function, first connect to the Management
Module from a Telnet or SSH client. You are prompted to enter a user ID and
password. The default user ID is USERID and the default password is
PASSW0RD, where 0 in the default password is a zero. Consider changing the
defaults in a production environment.
There are several different commands that you can use to access the SoL
remote text console function. The most useful commands are listed in Table 5-2.
Table 5-2 Management Module commands for SoL
Command What it does
console Open a SoL remote text console for the blade server. This
command fails if another SoL remote text console is already open
for the blade server.
console -o Terminate any existing SoL remote text console for the blade server
and open a SoL remote text console for the blade server.
boot -c Reset the blade server, and then open a SoL remote text console
for the blade server.
reset -c This is functionally equivalent to the boot -c command when used
in a blade server context.
power -on -c Power on the blade server and then open a SoL remote text
console for the blade server.
power -cycle -c Power on the blade server and then open a SoL remote text
console for the blade server. If the blade server is already powered
on, power it off first, and then power on.
Here are some examples of how to use these commands. To open a SoL remote
text console to the blade server in bay 3, you use the following command:
console -T system:blade[3]
To reset the blade server in bay 2 and then open a SoL remote text console to the
blade server, you use the following command:
boot -c -T system:blade[2]
To terminate an active SoL remote text console, press the Esc key followed by an
open parenthesis (Shift+9 on U.S. keyboards). When the SoL remote text
console ends, you return to the Management Module CLI prompt.
Inactivity timeout
The Management Module CLI has an aggressive default inactivity timeout. If you
do not use the CLI or a SoL remote text console accessed through the CLI for 2
Chapter 5. Hardware setup 93

minutes, you are disconnected. You can increase the timeout using the
telnetcfg command. For example, the following command increases the
inactivity timeout to 10 minutes (600 seconds):
telnetcfg -t 600 -T system:mm[1]
Setting CRLF
Over the years, there have been many variations on the interpretation of the
Enter key in a Telnet session. Some Telnet daemons expect carriage return +
linefeed (CRLF) and some expect carriage return alone (CR). From our testing,
CR seems to be the most compatible. During your SoL session, if when you
press the Enter key, the key behaves as though it was pressed twice and not
once, then correct the CRLF option. Under Windows 2000 Telnet, do this as
shown in Figure 5-15 before you open a connection to the Management Module.
Microsoft (R) Windows 2000 (TM) Version 5.00 (Build 2195)
Welcome to Microsoft Telnet Client
Telnet Client Build 5.00.99206.1
Escape Character is 'CTRL+]'
Microsoft Telnet> unset crlf
Microsoft Telnet> open bcmm
Figure 5-15 Setting CRLF in Windows 2000 Telnet
Refer to the developer of your client for other Telnet implementations.
5.5.3 Open Firmware interfaces
In some situations, you may need to access the Open Firmware interfaces to the
BladeCenter JS20 firmware. This section explains how to access Open Firmware
interfaces and how to use some of the functions they provide.
Activating the Open Firmware interfaces
You can access the Open Firmware interface whenever you power on a
BladeCenter JS20. Before you can access the Open Firmware interface, set up
the SoL remote text console function as described in 5.5, “Providing a console for
the JS20 blades” on page 87.
Use the following procedure to access the Open Firmware interface:
1. Using a Telnet or SSH client, connect to the Management Module external
Ethernet interface IP address.
94 The IBM Eserver BladeCenter JS20

2. When prompted, enter a valid user ID and password. The default
Management Module user ID is USERID, and the default password is
PASSW0RD, where the 0 in the default password is a zero. Consider
changing the defaults in any production environment.
3. Power cycle the blade and start an SoL console by using the power -cycle -c
command. For example, to power cycle and start a SoL remote text console
with the blade server in the first bay (bay 1), use the command:
power -cycle -c -T system:blade[1]
4. After approximately 30 seconds, you see a sequence of checkpoint codes
displayed on the console. These codes are generated by the Power On Self
Test (POST). The meaning of the checkpoint codes is documented in the
IBM Eserver BladeCenter JS20 Type 8842 Installation and User’s Guide.
5. Wait until the checkpoint code D5BB is displayed and then type 8. The POST
pauses for approximately five seconds at this checkpoint code to give you
time to respond to the checkpoint code. A number is displayed after the D5BB
checkpoint code, indicating how many seconds remain before the POST will
proceed further. For example, Figure 5-16 shows that four seconds remain.
E153 U8842.P1Z.23A1073-P1-T7
EAA1 U8842.P1Z.23A1073-P1
E172 U8842.P1Z.23A1073-P1
EAA1 U8842.P1Z.23A1073-P1
E152 U8842.P1Z.23A1073-P1
E153 U8842.P1Z.23A1073-P1
E152 U8842.P1Z.23A1073-P1
E153 U8842.P1Z.23A1073-P1
EAA1 U8842.P1Z.23A1073-P1
E172 U8842.P1Z.23A1073-P1
EAA1 U8842.P1Z.23A1073-P1-C5
D001
D003
D004
E139
E14A
D008
E1F0
E1F1
D099
D5BB 4
Figure 5-16 POST checkpoint codes
Chapter 5. Hardware setup 95

6. As soon as you type 8, the Open Firmware command prompt is displayed.
If you do not type any key when the D5BB checkpoint code is displayed, the
blade server proceeds to boot using the default boot sequence. To learn how to
configure the boot sequence, see 5.3, “Blade server configuration” on page 76.
5.5.4 Specifying IP parameters to Open Firmware
You can use a directed bootp to install a JS20 blade from a NIM server. bootp
does not require the NIM server to be on the same subnet as the JS20 blade.
Nor does this option require that you have the MAC address of the network
adapter on the JS20 blade.
To perform a directed bootp, you need an SoL connection to the blade so that
you can specify the IP parameters to Open Firmware. Currently you must have
two network adapters to perform a NIM installation if you are using SoL. You
cannot install AIX 5L over the same adapter that is using SoL.
Step 1: Preparing your NIM server
Follow these steps to prepare your NIM server.
1. Create a SPOT, lpp_source, and any other resources that you need at the
level of AIX 5L that you want to install on your NIM server. Your NIM server is
usually the NIM master, but you can also set up a NIM client as a NIM server.
For instructions on how to create NIM resources, see 7.1, “Minimal NIM
installation” on page 124.
2. Ensure that you have the information in the following worksheet (Table 5-3) for
your JS20 blade before you proceed with the installation.
Table 5-3 Network configuration information worksheet
Network attribute Value
Network interface
Host name
IP address _______.________.________.________
Network mask _______.________.________.________
Name server _______.________.________.________
Domain name _______.________.________.________
Gateway _______.________.________.________
96 The IBM Eserver BladeCenter JS20

3. Define the JS20 blade as a NIM client on your NIM master by running the
following command on the NIM master:
smitty nim_mkmac
This command creates a client definition for your JS20 blade. You can also
define the JS20 blade using the define NIM operation on the command line.
4. If you want to set the JS20 blade's name server and domain name after the
installation, use a resolv_conf resource.
5. Set up your NIM master to install the JS20 blade, by running this command:
smitty nim_bosinst
Select the JS20 blade that you defined earlier as your target. Then select the
type of install that you want to perform and select the installation resources
that you want to use to install the JS20 blade. You can also prepare the JS20
blade to install using the bos_inst NIM operation on the command line.
Note: If the JS20 blade is powered off or was never installed, set “Initiate
reboot and installation now?” to no and press Enter in the SMIT interface.
If the JS20 blade is powered on and running AIX 5L, set “Initiate reboot
and installation now?” to yes in the SMIT interface. If you choose this
option, a directed bootp is initiated by default and you can skip step 2.
Before you run this command, ensure that the JS20 blade is a registered
NIM client.
1. Run smitty niminit on the JS20 blade.
2. Specify the host name of your NIM master and the interface you want to
use for the installation.
You can also initialize the JS20 blade using the niminit command on the
command line.
Step 2: Specifying a directed bootp from the JS20 blade
Specify a directed bootp from the JS20 blade by using these steps:
1. Open a Web interface to the Management Module by navigating to the IP
address or host name of the Management Module using a Web browser.
2. Enable SoL to the JS20 blade from the Management Module Web interface.
Select Blade Tasks →Serial Over LAN.
3. Select the JS20 blade that you are installing and click Enable Serial Over
LAN.
4. Power on the JS20 blade from the Management Module Web interface Blade
Tasks →Power/Restart.
Chapter 5. Hardware setup 97

5. Select the JS20 blade that you are installing and click Power On Blade.
6. Open a Serial over LAN connection to the JS20 blade by using Telnet into the
Management Module and running the console command. For example, if the
JS20 blade is in slot 3, run the following command:
console -T blade[3]
The Serial over LAN connection shows a series of LED numbers.
7. Type 8 when you see D5BB to go to the Open Firmware prompt.
8. Boot from the network by typing:
boot net:bootp,server_ip,,client_ip,gateway_ip
Because SoL uses ent0, you must use ent1. Run a command similar to this
one (note the double comma):
boot
/pci@8000000f8000000/pci@0/ethernet@1,1:bootp,192.168.2.10,,192.168.1.11,192.168.1.1
98 The IBM Eserver BladeCenter JS20
Note: You must specify the full device path name with this command. To
determine the full path to your device, list the device tree by running the ls
command at the Open Firmware prompt. This command displays output
similar to this:
0 > ls
000000c87f18: /ibm,serial
000000c88840: /chosen
000000c88a98: /packages
...
000000d31488: /vdevice
000000d327a8: /vty@0
000000d32f88: /IBM,sp@4000
000000d33f10: /rtc@4001
000000d34a18: /pci@8000000f8000000
000000d384d0: /pci@0
000000d4bbd0: /ethernet@1
000000d5af50: /ethernet@1,1
000000d3be00: /pci@3
000000d6a350: /usb@0
000000d845f8: /hub@1
000000d854b8: /usb@0,1
000000d9f760: /hub@1
000000d3f798: /pci@1f
000000d45ed8: /ide@4,1
000000d47b10: /disk@0
The highlighted items are the path to the second Ethernet adapter. You
pass this information to the boot command to initiate a network boot from
the second Ethernet adapter.

After you run the boot command, the network installation begins. Output similar
to Example 5-1 is displayed on the SoL connection.
Example 5-1 Output of the boot command
BOOTP: chosen-network-type =
ethernet,auto,none,auto
BOOTP: server IP = 192.168.2.10
BOOTP: requested filename =
BOOTP: client IP = 192.168.1.11
BOOTP: client HW addr = 0 d 60 1e c cb
BOOTP: gateway IP = 192.168.1.1
BOOTP: device
/pci@8000000f8000000/pci@0/ethernet@1,1
BOOTP: loc-code U8842.P1Z.23A0984-P1-T7
BOOTP R = 1
FILE: /tftpboot/js20blade1.austin.ibm.com
Load Addr=0x0000000000004000, Max
Size=0x0000000000bfc000
FINAL Packet Count = 21131
FINAL File Size = 10818623 bytes.
load-base=0x4000
real-base=0xc00000
Elapsed time since release of system
processors: 2 mins 28 secs
Chapter 5. Hardware setup 99

100 The IBM Eserver BladeCenter JS20

Chapter 6. Installing Linux
6
This chapter explains how to install Red Hat Enterprise Linux (RHEL) and SUSE
LINUX Enterprise Server (SLES) on the IBM Eserver BladeCenter JS20.
© Copyright IBM Corp. 2005. All rights reserved. 101

6.1 Installing Linux
Both Red Hat Enterprise Linux and SUSE LINUX Enterprise Server are
supported operating systems for the BladeCenter JS20. This chapter explains
how to set up and conduct a network-based installation of both distributions.
Because the general process of doing this is similar for both operating systems,
the focus of this chapter is on the installation of Red Hat Enterprise Linux 3.0 and
SUSE LINUX Enterprise Server 9.0.
Tip: Prior to installing any operating system, we recommend that you upgrade
all firmware to the latest level. See 5.4, “Firmware” on page 79, for instructions
about how to complete this task.
6.1.1 Configuring the sources
Since we are doing a network installation, the first step is to configure a file space
that contains the source files for the operating system installation. For all cited
examples, we created the sources on a pSeries 690 that was connected to the
same local area network (LAN) as the BladeCenter. Since there are special tools
available within each distribution for this purpose, we configured each network
install server with a similar distribution. For example, the server that installed
RHEL 3 also ran RHEL 3.
Tip: We recommend a 100 Mb or faster LAN that is local to both the machine
that will host the files and the BladeCenter. We also recommend that the file
spaces be shared via Network File System (NFS).
Red Hat sources
The Red Hat installer, Anaconda, makes it simple to set up a remote file share.
Simply copy the ISOs for the source CDs into your NFS mount. To create an ISO
of the first CD and place it in /mnt/exports/, enter:
# dd if=/dev/cdrom of=/mnt/exports/RHEL-3.0-U2-disc1.iso bs=32M
Repeat the previous step for all the CDs, changing the file name as appropriate.
You may also increase the bs parameter as appropriate. This parameter controls
block size. The larger the block size is, the more RAM is taken for the dd process,
but the faster the process takes.
Important: Ensure that the CD is not mounted before you begin the dd. Also
ensure that the destination of the ISO has enough space to store all the data.
Copying a full complement of RHEL 3 ISOs can take over 4 GB.
102 The IBM Eserver BladeCenter JS20

Configure the NFS daemon to export the mount point where the ISOs are
located. You can do this by editing /etc/exports. Example 6-1 shows the export
file we used.
Example 6-1 /etc/exports file contents
/mnt/exports/ 192.168.1.0/24(ro)
After the /etc/exports file is created or updated, we enabled NFS as shown in
Example 6-2.
Example 6-2 Starting NFS
# service nfs start
Starting NFS services: [ OK ]
Starting NFS quotas: [ OK ]
Starting NFS daemon: [ OK ]
Starting NFS mountd: [ OK ]
#
Attention: If NFS does not start properly, refer to the Red Hat Enterprise
Linux Reference Guide at the following Web site for troubleshooting
information:
http://www.redhat.com/docs
Chapter 6. Installing Linux 103

SuSE sources
SuSE installations are usually administered via a program called Yet Another
Setup Tool (YaST).
1. Within YaST, there is an applet for configuring installation sources on the
server. Access this from the Misc section in the left pane, as shown in
Figure 6-1.
Figure 6-1 YaST installation server configuration
104 The IBM Eserver BladeCenter JS20

2. You now see the Source Configuration panel (Figure 6-2). Select SuSE SLES
Version 9 (ppc).
Figure 6-2 Selecting the distribution
Chapter 6. Installing Linux 105

3. In the next panel as shown in Figure 6-3, you select which media to use as a
source for the SuSE distribution. We found it easier to use ISOs rather than
CDs. Proceeding past this window builds the source directory. You may be
prompted for additional CDs (or locations of ISOs) while the tree is being built.
Tip: If you have any service packs or additional CDs that you want to add,
select the Prompt for Additional CDs check box.
Figure 6-3 Source Configuration pane
106 The IBM Eserver BladeCenter JS20

4. After the source files are copied into the installation share, click Finish as
shown in Figure 6-4. This completes any final tasks such as starting the NFS
daemon.
Figure 6-4 Completing the setup
6.1.2 The zImage.initrd file
PowerPC-based hosts support booting directly from the network provided that
they are served a zImage.initrd via BOOTP. The zImage.initrd is a package that
contains both the kernel and initial RAM disk in a single file.
Most distributions already have a zImage.initrd, which you can use for network
booting. For RHEL, you can find this file on the first CD as images/netboot.img.
For SuSE, this file is called install and is in the root directory of the first CD.
Although these files are referred to by different names, they are the zImage.initrd
file that we require.
Building your own zImage.initrd
Although most users use the zImage.initrd provided by their distribution, you can
also compile your own.
Tip: This section address the nuances of how to create a zImage.initrd. If you
need more information about compiling a kernel, see the Kernel HOWTO at:
http://www.tldp.org
Chapter 6. Installing Linux 107

To build your own zImage.initrd, follow these steps:
1. Obtain the necessary kernel sources and development utilities for your
distribution as appropriate. If the necessary packages are not installed, you
may experience some errors, which cause your compilation to fail. Consult
your distribution’s documentation to determine the names of these packages.
2. With the packages installed, start by cleaning the source tree. This is a good
procedure to ensure that there are no remnants of an old compile left hanging
around. Here is an example of how to do this.
# make mrproper
3. With the source tree clean, configure your kernel as required. The preferred
method to do this is to use either make menuconfig (text based) or make
xconfig (graphical; requires an X server).
Both Red Hat and SuSE have their default kernel options selected when
menuconfig / xconfig is launched after having done a make mrproper. From
either interface, select the desired options.
4. After you configure the kernel, perform the compilation. Since RHEL 3 is
based on the 2.4 kernel and SLES 9 is based on 2.6, the next steps are
slightly different. Refer to Example 6-3 and Example 6-4 for each distribution.
Example 6-3 Compiling a kernel in Red Hat
# make dep
# make clean
# make zImage
# make modules
# make modules_install
# make install
Example 6-4 Compiling a kernel in SuSE
# make
# make modules_install
# make install
108 The IBM Eserver BladeCenter JS20

5. After you compile the kernel, create the initial RAM disk. Both Red Hat and
SuSE ship with a command called mkinitrd for this purpose. However, the
syntax for each is slightly different, as shown in Example 6-5 and
Example 6-6.
Example 6-5 mkinitrd for Red Hat
# cd arch/ppc64/boot
# mkinitrd -f ramdisk.image 2.4.21-15.ELcustom
# gzip ramdisk.image
# cd /usr/src/linux-2.4
# make zImage.initrd
Example 6-6 mkinitrd for SuSE
# make install
# mkinitrd
# cd arch/ppc64/boot/
# cp /boot/initrd-2.6.5-7.39-pseries64 ramdisk.image
# gzip ramdisk.image
# cd /usr/src/linux
# make zImage.initrd
Attention: The file names referred to are all relative to the kernel version
strings within the Makefile for the kernel. The files on your system may be
different.
After you complete these steps, you should have a bootable zImage.initrd file in
/boot/ppc64/.
6.2 Configuring BOOTP and TFTP
There are three steps to network booting a POWER/PowerPC-based server.
1. Perform a Power On Self Test (POST) and then receive BOOTP information
from BOOTP server, including the IP, Trivial File Transfer Protocol (TFTP)
server IP, and zImage.initrd file name.
2. TFTP server sends the zImage.initrd file.
3. The server boots using the zImage.initrd file.
Both RHEL and SLES ship with Dynamic Host Configuration Protocol (DHCP)
and TFTP packages. Ensure that you have installed the appropriate RPMs.
Chapter 6. Installing Linux 109

6.2.1 BOOTP
DHCP is an extension to the original BOOTP specification. As a result, you can
use the DHCP daemon (dhcpd) to provide the BOOTP information for booting via
the network.
The DHCP daemon is configured through the /etc/dhcpd.conf file. Example 6-7
shows the configuration that we used in our lab.
With dhcpd configured, start the DHCP daemon as shown here:
# /etc/init.d/dhcpd start
Example 6-7 /etc/dhcpd.conf
always-reply-rfc1048 true;
allow bootp;
deny unknown-clients;
not authoritative;
default-lease-time 600;
max-lease-time 7200;
subnet 192.168.1.0 netmask 255.255.255.0 {
group {
next-server 192.168.1.1;
}
Attention: Keep in mind that Example 6-7 contains IP and MAC address
information specific to our lab environment. See your product documentation
or the dhcpd man page to customize this for your site.
}
110 The IBM Eserver BladeCenter JS20
host blade1 {
fixed-address 192.168.1.117;
hardware ethernet 00:0d:60:1e:0f:89;
filename "zImage.initrd.suse";
}
host blade2 {
fixed-address 192.168.1.118;
hardware ethernet 00:0d:60:1e:0e:fb;
filename "zImage.initrd.redhat";
}

6.2.2 Trivial File Transfer Protocol
The TFTP daemon for both distributions considers the /tftpboot directory its
home. Any file name statements made in the /etc/dhcpd.conf configuration file
should be relative to this path.
Both RHEL and SLES spawn TFTP from the xinetd daemon. By default, tftpboot
is disabled on both distributions. To enable both xinetd and TFTP, enter:
[root@blade1 root]# chkconfig tftp on
[root@blade1 root]# /etc/init.d/xinetd restart
Your installation server is now ready to network boot the BladeCenter JS20.
6.3 Preparing an unattended install
RHEL and SLES both have their own methods of conducting an unattended
installation. RHEL uses Kickstart, and SLES uses a method known as AutoYaST.
Chapter 6. Installing Linux 111

6.3.1 Red Hat Kickstart
Red Hat provides a utility called redhat-config-kickstart to assist with the
creation of Kickstart files. When you launch this utility, you see the Kickstart
Configurator window (Figure 6-5).
Figure 6-5 redhat-config-kickstart main window
The separate sections of this application allow you to specify different attributes
of the installation.
► Basic Configuration
You can configure language support, keyboard, mouse, and root password
here. You may also specify whether you prefer a text installation instead of a
graphical installation. Lastly you may choose if the system should reboot after
the automated installation is complete.
► Installation Method
You can choose an upgrade or clean installation here. An upgrade usually
allows the preservation of existing data by upgrading the installed applications
112 The IBM Eserver BladeCenter JS20

from a previous version of Red Hat. You can also specify the source of the
installation (NFS, HTTP, CDROM, etc.) here.
► Boot Loader Options
Select the type of bootloader (GRUB or LILO) along with any relevant options
from here. You may also specify kernel parameters from this window.
► Partition Information
You specify the partition size and type information here. You also specify the
action to take with existing partitions and new disks.
► Network Configuration
Define population network information, such as IP, DNS, and routers, from
here.
► Authentication
You may configure NIS, Keberos, and other authentication server options
here. You may also specify local password encryption and security.
► Firewall Configuration
Configure iptables rules from here.
► Xconfiguration
Select the video card, resolution, monitor, and window manager here.
► Package Selection
Select which programs are installed on the system here.
► Pre-Installation Script
Identify the commands to run before the installation.
► Post-Installation Script
Identify the commands to run after the installation.
When all your options are specified, save the file. This automatically checks for
any missing options. If everything is correct, you may now use the ks.cfg file in an
unattended installation.
Tip: We found that the redhat-config-kickstart program, by default, places a
skipx parameter in the ks.cfg file. We had to comment out this prior to using
ks.cfg. Otherwise, the installation would fail.
Chapter 6. Installing Linux 113

6.3.2 SuSE AutoYaST
SLES provides a YaST applet to assist with the creation and editing of AutoYAST
files. You can launch it from the Misc section of YaST, as shown in Figure 6-6.
Figure 6-6 AutoYaST module within YaST
114 The IBM Eserver BladeCenter JS20

After the AutoYaST editor is created, you see a window similar to the one in
Figure 6-7.
Figure 6-7 Main AutoYaST window
You can use the following sections to configure your AutoYaST file.
► Software
Select the packages to install, and configure the automatic update client.
► Hardware
Configure partitioning, audio, printing, and X.
Chapter 6. Installing Linux 115

► System
Set the general system information. You also configure the language, time
zone, and other locale-related settings here. And specify boot loader, logging,
and runlevel information here.
► Network Devices
Set network adapter information. Kernel module information as well as IP
details can be set here.
► Network Services
Network clients and daemons can be configured from here. There are over 15
different daemons to choose from. Refer to the Network Services section for
more information.
► Security and Users
From here, you may create users that you want to be present after install. You
may also change the security policy to make things such as password
changes more stringent. And you can configure VPN and firewall settings
from here.
► Misc
This section allows you to add complete configuration files, or to add special
scripts to run before and after installation.
Tip: AutoYaST also lets you import Kickstart files from Red Hat or Alice (a
precursor to AutoYaST) files. You do this by selecting File → Import.
After you are satisfied with the selections, save the file. This validates that the
required minimum options are populated prior to saving.
6.4 Performing an unattended installation
The unattended installation process needs to pass the zImage.initrd the location
of the unattended answer file during boot. Unfortunately, BOOTP does not have a
facility to provide anything more than the location to the zImage.initrd and the
tftpboot server to the client. To pass the required parameters, they need to be
input into Open Firmware.
116 The IBM Eserver BladeCenter JS20

6.4.1 Open Firmware
For instructions about accessing the Open Firmware prompt, refer to “Activating
the Open Firmware interfaces” on page 94.
When you reach the Open Firmware prompt, pass bootparm through the boot-file
variable. Example 6-8 and Example 6-9 show the settings for SuSE and Red Hat
respectively.
Example 6-8 Open Firmware settings for SuSE
0 > setenv boot-file autoyast=nfs://192.168.1.113:/mnt/SUSE-Source/autoyast.xml
install=nfs://192.168.1.113:/mnt/SLES-Source/SLES9/ netdevice=eth1
Example 6-9 Open Firmware settings for Red Hat
0 > setenv boot-file ks=nfs:192.168.1.112:/mnt/install-images/ks.cfg
ksdevice=eth1
After the installation is complete, remember to clear the boot-file value. If this
value is not cleared, it is passed to the kernel as bootparm next time you boot.
You can clear the value by using the nvsetenv command. We recommend that
you run this command after the installation completes via either AutoYaST or
Kickstart, as appropriate.
This command is provided by the ppc64-utils RPM in RHEL and the util-linux
RPM in SLES. The syntax to clear the boot-file variable is:
# /sbin/nvsetenv boot-file ""
6.4.2 mkzimage_cmdline: SuSE only
mkzimage_cmdline is a utility contained in the SLES 9 lilo RPM. The SuSE kernel
has special markers that allow mkzimage_cmdline to pass bootparm information
directly through the zImage.initrd. Example 6-10 shows the syntax.
Example 6-10 mkzimage_cmdline
# /lib/lilo/chrp/mkzimage_cmdline -a 1 -c -s
"autoyast=nfs://192.168.1.113:/mnt/SUSE-Source/autoyast.xmlinstall=nfs://192.16
8.1.113:/mnt/SLES-Source/SLES9/ netdevice=eth1" netboot.suse.img
# /lib/lilo/chrp/mkzimage_cmdline netboot.suse.img
cmd_line size:512
cmd_line: autoyast=nfs://192.168.1.113:/mnt/SUSE-Source/autoyast.xml
install=nfs://192.168.1.113:/mnt/SLES-Source/SLES9/ netdevice=eth1
active: 1
Chapter 6. Installing Linux 117

The syntax for mkzimage_cmdline is straightforward. When entered with only a file
name parameter. The current bootparm is displayed. You can use the following
options:
-h Display help.
-v Display version.
-a 0|1 Disable/enable command line. Overrides bootparm passed from
Open Firmware.
-s STRING Stores STRING as bootparm in zImage.initrd.
-c Clears previously stored bootparm from zImage.initrd before
applying a new one.
6.5 Performing a network installation
When installing several machines, we recommend that you perform a network
installation instead of a CD-based installation. The speed and ability to serve
multiple machines makes it ideal for installing to a full complement of
BladeCenter JS20s.
Attention: The cited examples are provided for illustration purposes only. For
detailed instructions about how to install each distribution, refer to the product
documentation.
6.5.1 SUSE LINUX Enterprise Server 9
For the SUSE LINUX Enterprise Server 9, follow this procedure:
1. After you load the zImage.initrd file, you see a language display similar to the
one in Figure 6-8. In this example, we selected English.
118 The IBM Eserver BladeCenter JS20

Select the language.
1) Bosnia
2) Cestina
3) Deutsch
4) English
5) Español
6) Français
7) Hellenic
8) Italiano
9) Japanese
10) Magyar
11) Nederlands
12) Polski
13) Português
14) Português Brasileiro
15) Russian
16) Slovencina
>
Figure 6-8 Language selection
2. After you select the language, the main installer display appears with a
number of options. In this example, we chose to load additional kernel
modules, as shown in Figure 6-9.
>>> Linuxrc v1.6 (Kernel 2.6.5-7.69-pseries64) (c) 1996-2004 SUSE LINUX AG

3. We select the install option and start the installation, as shown in Figure 6-10.
Start Installation or System
1) Start Installation or Update
2) Boot Installed System
3) Start Rescue System
>
Figure 6-10 Installation menu
4. We instruct the installer to use the network as a source for the installation, as
shown in Figure 6-11.
Choose the source medium.
1) CD-ROM
2) Network
3) Hard Disk
>
Figure 6-11 Source menu
5. We chose NFS as the protocol to use from the menu in Figure 6-12.
Choose the network protocol.
1) FTP
2) HTTP
3) NFS
4) TFTP
>
Figure 6-12 Protocol menu
120 The IBM Eserver BladeCenter JS20

6. We chose the following site-specific information. You may have different
information for your environment, but the general process is the same as
shown in Figure 6-13.
Automatic configuration via DHCP?
1) Yes
2) No
> 1
Sending DHCP request...
Enter the IP address of the NFS server [192.168.1.113]>
Enter the directory on the server
[/mnt/SLES/SLES9-RC1/]>
Figure 6-13 Installation location
With these steps complete the normal YaST process begins.
Tip: With this, the network specific portion of the installation is complete. Refer
to the product documentation for further information.
6.5.2 Red Hat Enterprise Linux 3
After the zImage.initrd is served by the TFTP boot server, the installation process
is the same as a CD-based installation. Refer to the product documentation for
more information regarding this.
Chapter 6. Installing Linux 121

122 The IBM Eserver BladeCenter JS20

Chapter 7. Installing AIX on the JS20
7
This chapter discusses NIM. It explains how to configure NIM to install AIX and
how to install AIX on an IBM Eserver BladeCenter JS20 using NIM.
BladeCenter JS20 supports the installation of AIX 5.2f and later. We recommend
that you install AIX using a network installation via NIM. NIM is a unified console
that allows the management of several AIX logical partitions (LPARs) or
machines. You may install AIX on new machines or manage existing ones from a
unified interface.
Tip: Prior to installing any operating system, we recommend that you upgrade
all firmware to the latest level. See 5.4, “Firmware” on page 79, for instructions
about how to complete this task.
To use NIM, there should be an available AIX instance with NIM configured on
the same network segment as the BladeCenter. In the interest of speed and
security, a private dedicated subnet is recommended.
Our NIM master was configured on a pSeries 690 connected to the same local
area network (LAN) as the BladeCenter. This p690 was running the same level of
AIX that we installed on the BladeCenter JS20.
Note: An installation of AIX may be several gigabytes in size. We recommend
that you use a 100 Mb or faster network for NIM-based installations.
© Copyright IBM Corp. 2005. All rights reserved. 123

7.1 Minimal NIM installation
Follow these steps to install NIM:
1. Launch the Web-based Systems Manager (WebSM) and point to the server
that is running NIM. Select Configure the Network Installation
Management environment from the NIM object shown in Figure 7-1.
Figure 7-1 NIM configuration window
2. In the window that opens (Figure 7-2), select Configure this host as NIM
Master.
124 The IBM Eserver BladeCenter JS20

Figure 7-2 Configuring as a NIM Master
3. Select the network interface that is bound to the same network as the
BladeCenter, and name it appropriately, as shown in Figure 7-3.
Figure 7-3 Selecting the NIM interface and network name
Chapter 7. Installing AIX on the JS20 125

4. In the next window (Figure 7-4), select Initialize Minimal NIM Environment.
Figure 7-4 Choosing to initialize minimal NIM environment
5. Click View Settings to confirm the gathered settings and complete the
minimal NIM environment as shown in Figure 7-5.
Figure 7-5 Finalizing the NIM environment settings
126 The IBM Eserver BladeCenter JS20

7.2 Adding resources
Follow these steps to add resources:
1. With the minimal NIM environment complete, configure lpp_sources and a
SPOT on the NIM master. Return to the NIM configuration window within
WebSM and select Add a new resource to the Network Installation
Management environment.
2. In the Add New Resource window (Figure 7-6), select the lpp_source &
SPOT option.
Figure 7-6 lpp_source and SPOT configuration
Chapter 7. Installing AIX on the JS20 127

3. In the Configure NIM Environment window (Figure 7-7), select the medium
that you will use for your AIX source. We use the DVD RAM drive.
Figure 7-7 AIX source
4. As shown in Figure 7-8, select the default values and begin populating the
NIM resources.
Figure 7-8 Default resource settings
128 The IBM Eserver BladeCenter JS20

5. Click View Settings, as shown in Figure 7-9, to confirm the settings.
Assuming a CD-based installation, insert the first CD and proceed to
complete the addition of the new resources.
Figure 7-9 Completing the resource installation
Tip: This step make take some time to complete, since many files must be
incorporated into the resource.
Chapter 7. Installing AIX on the JS20 129

7.3 Adding machine information
With the resources added, add information regarding the machine on which you
want to perform a base operating system (BOS) installation.
1. From the main NIM menu (Figure 7-10), select Add a new Machine to the
Network Installation Management environment.
Figure 7-10 Adding new machine information to NIM
130 The IBM Eserver BladeCenter JS20

2. In the Add New Machine window (Figure 7-11), enter the host name of the
server that you want to add to the NIM environment from this window.
Figure 7-11 Host name of new machine
Tip: The NIM server must be able to resolve the host name of the new
server. Ensure that this host name can be resolved by DNS or a hosts file.
Chapter 7. Installing AIX on the JS20 131

3. In the next window (Figure 7-12), select Multiprocessor and confirm the
other metrics of the machine that you are adding to the NIM environment.
Figure 7-12 Entering machine-specific information
132 The IBM Eserver BladeCenter JS20

4. As shown in Figure 7-13, enter the MAC address for the machine that you are
adding to the Network adapter hardware address field and proceed to
complete the addition of the new machine to the NIM environment.
Figure 7-13 Entering MAC address information
With the NIM resources created and the machine object created, simply proceed
with the installation.
NIM allows you to install new operating systems on pSeries servers by booting
the machine via BOOTP. This diskless or network installation makes it simple to
install new operating systems to servers in a matter of minutes.
Chapter 7. Installing AIX on the JS20 133

7.4 Preparing the NIM master
With all the NIM resources in place, it is simple to begin the installation of a new
operating system to the target machine, in this case a BladeCenter JS20.
1. Launch WebSM and view the configured machine objects from the NIM
section.
2. Right-click the object for the desired machine and select Install Operating
System, as shown in Figure 7-14.
Figure 7-14 Installing an operating system from NIM
3. Select the location of the SPOT and lpp_source resources created earlier.
134 The IBM Eserver BladeCenter JS20

4. Proceed with the installation by setting the BOS preferences as shown in
Figure 7-15.
Figure 7-15 Setting BOS preferences
Tip: To fully automate the AIX installation, create a bosinst_data resource and
associate it with the operating system installation in NIM settings. Although
this is beyond the scope of this book, you can find instructions in the AIX
product manuals.
7.5 Preparing the client
With the NIM server configuration complete, simply boot the BladeCenter JS20
from the network as explained in 5.5.3, “Open Firmware interfaces” on page 94.
When the NIM master supplies the boot image via TFTP, the AIX installation
begins just as though you booted from the CD.
Chapter 7. Installing AIX on the JS20 135

1. On the first panel, select the desired language. We selected English, as
shown in Figure 7-16.
******* Please define the System Console. *******
Type a 1 and press Enter to use this terminal as the
system console.
Pour definir ce terminal comme console systeme, appuyez
sur 1 puis sur Entree.
Taste 1 und anschliessend die Eingabetaste druecken, um
diese Datenstation als Systemkonsole zu verwenden.
Premere il tasto 1 ed Invio per usare questo terminal
come console.
Escriba 1 y pulse Intro para utilizar esta terminal como
consola del sistema.
Escriviu 1 1 i premeu Intro per utilitzar aquest
terminal com a consola del sistema.
Digite um 1 e pressione Enter para utilizar este terminal
como console do sistema.
Figure 7-16 Installation language panel
2. In the next panel (Figure 7-17), you may proceed with the default installation
options or customize them. We chose to customize them.
Welcome to Base Operating System
Installation and Maintenance
Type the number of your choice and press Enter. Choice is indicated by >>>.
>>> 1 Start Install Now with Default Settings
Figure 7-17 Operating system selection
136 The IBM Eserver BladeCenter JS20
2 Change/Show Installation Settings and Install
3 Start Maintenance Mode for System Recovery
88 Help ?
99 Previous Menu
>>> Choice [1]:

3. In the Installation and Settings window (Figure 7-18), we proceeded with the
default installation type and language environment values.
Installation and Settings
Either type 0 and press Enter to install with current settings, or type the
number of the setting you want to change and press Enter.
1 System Settings:
Method of Installation.............Preservation
Disk Where You Want to Install.....hdisk0
2 Primary Language Environment Settings (AFTER Install):
Cultural Convention................C (POSIX)
Language...........................C (POSIX)
Keyboard...........................C (POSIX)
3 More Options (Desktop, Security, Kernel, Software, ...)
>>> 0 Install with the settings listed above.
+-----------------------------------------------------
88 Help ? | WARNING: Base Operating System Installation will
99 Previous Menu |destroy or impair recovery of SOME data on the
|destination disk hdisk0.
>>> Choice [0]:
Figure 7-18 Installation and language options
Chapter 7. Installing AIX on the JS20 137

4. As illustrated in Figure 7-19, we elected to include additional options such as
the 64-bit kernel and JFS2. We then selected 0 and began the installation
process.
Install Options
1. Enable Trusted Computing Base.................................... no
2. Enable CAPP and EAL4+ Technology................................. no
(English only, 64-bit kernel enablement, JFS2 file systems)
3. Enable 64-bit Kernel............................................. yes
4. Create JFS2 File Systems......................................... yes
5. Graphics Software................................................ yes
6. Documentation Services Software.................................. no
7. Enable System Backups to install any system...................... no
(Installs all devices and kernels)
>>> 8. Install More Software
Figure 7-19 Kernel options
138 The IBM Eserver BladeCenter JS20
0 Install with the settings listed above.
88 Help ?
99 Previous Menu
>>> Choice [8]:
Tip: As with any operating system, AIX enables you to customize many
different options during the installation process. We have provided a sample of
what you can select.

Chapter 8. System management
scenarios
8
Your installation may be comprised of a single BladeCenter chassis with only a
few BladeCenter JS20s. If so, you may find that the capabilities provided by the
BladeCenter Management Module Web interface and command line interface
(CLI) are sufficient to meet your system management requirements. For larger
installations comprised of multiple BladeCenter chassis and many BladeCenter
JS20s, consider using IBM Director, IBM Cluster Systems Management (CSM),
or both.
We demonstrate how to use many of the functions provided by the BladeCenter
Management Module Web interface and CLI in Chapter 5, “Hardware setup” on
page 69.
This chapter provides information about setting up and configuring the IBM
Director and the IBM Cluster Systems Management facilities. It also describes
several systems management scenarios for the BladeCenter JS20.
© Copyright IBM Corp. 2005. All rights reserved. 139

8.1 IBM Director
IBM Director is a comprehensive systems-management solution. A powerful
suite of tools and utilities, IBM Director automates many of the processes
required to manage systems proactively, including preventive maintenance,
diagnostic monitoring, troubleshooting, and more. It offers a graphical user
interface (GUI) that provides system administrators easy access to both local
and remote systems. IBM Director can be used in environments with multiple
operating systems (heterogeneous environments).
At the time of writing, the current version of IBM Director was Version 4.2.
IBM Director is the industry-leading client/server workgroup manager. IBM
Director’s tools provide customers with flexible capabilities to realize maximum
system availability and lower IT costs. With IBM Director, IT administrators can
view and track the hardware configuration of remote systems in detail and
monitor the usage and performance of critical components, such as processors,
disks, and memory.
Extensions to IBM Director are available for clients who want advanced
capabilities. These extensions include:
► Server Plus Pack
► Software Distribution Premium Edition
► Remote Deployment Manager
► Application Workload Manager
Some of the features that you find in IBM Director include:
► Self-managing, smart tools that are automated, and proactive capabilities that
reduce IT costs and maximize uptime
► Support for non-IBM hardware
Innovative use of industry standards from Common Information Model (CIM)
to Simple Network Mail Protocol (SNMP) enables heterogeneous hardware
management, protecting your existing IT investment.
► Seamless Integration
IBM Director protects your investments in other management packages,
complementing them with more extensive hardware manageability.
► Single-click management GUI
A convenient user interface delivers the ability to drag and drop tasks to
specific systems or groups of systems.
140 The IBM Eserver BladeCenter JS20

► Integrated, centralized SQL database
An internal database makes system-related data available, even when the
specific system is not directly available.
► Multiple operating system support
IBM Director smoothly handles a variety of popular operating systems.
► Comprehensive BladeCenter support
IBM Director provides an easy, single point of deployment and management
of new blade server architectures.
► Consistent framework that can be extended with plug-ins for advanced
management
► CLI support for many IBM Director management tasks
IBM Director 4.20 has many new features, support for new operating systems,
and enhancements to existing functions. Together, these changes provide
significantly richer and broader capabilities for superior hardware management.
Key enhancements include:
► Expanded Linux support including aggregated, color-coded systems health
status for systems running Linux
► Convenient subscription service included with the IBM Director license to
proactively deliver the latest product updates
► Enhanced optional Server Plus Pack, a collection of advanced tools to help
optimize server performance and availability
Some of the new features include XML report format for the Capacity
Manager and System Availability tools, the ability to schedule System
Availability reports, and Active PCI Manager support for Linux.
► The latest hardware support including support for additional hardware
industry standards for manageability, ASF 2.0 and IPMI, to further facilitate
the management of heterogeneous hardware environments
► A multitude of new features and refinements to make IBM Director even
easier to install and use
Chapter 8. System management scenarios 141

8.1.1 Setting up an IBM Director Server
Before Director clients can be installed, you must first configure the Director
server. The following instructions explain how we set up our server.
1. Mount the Director CD.
sysmgr:/ # mount -t iso9660 -o map=off /dev/cdrom /media/cdrom
Important: It is critical that the CD is mounted in this manner. If you do not
specify -o map=off, the capitalization of RPM names is not preserved and
the installation script fails.
2. With the CD mounted, copy the contents of the appropriate directory to the
hard drive of the server.
sysmgr:/root/Director-Source/ # cp
/media/cdrom/director/server/linux/i386/* .
3. With the sources copied to the hard drive, launch the installer.
sysmgr:/root/Director-Source/ # ./dirinstall
4. After the Director code is installed, configure a database for it to use. Because
PostgreSQL ships as a standard part of SLES, we chose to use it.
Example 8-1 shows how we configured it.
Example 8-1 Configuring PostgreSQL
sysmgr:/root/Director-Source/ # cd /usr/share/pgsql
sysmgr:/usr/share/pgsql/ # ln -s jdbc7.1-1.2.jar postgresql.jar
sysmgr:/usr/share/pgsql/ # cd /etc/TWGserver/
sysmgr:/etc/TWGServer/ # echo export CLASSPATH=/usr/share/pgsql/psotgresql.jar
> setup_env
sysmgr:/etc/TWGServer/ # chmod u+x setup_env
Important: We assume that PostgreSQL is configured and running
properly on the system. Refer to the documentation regarding how to do
this.
5. By default, PostgreSQL on SLES does not bind to a TCP/IP socket. To enable
this, modify /etc/sysconfig/postgresql and add the following line:
POSTGRES_OPTIONS="-i"
6. Begin the database configuration, as shown in Example 8-2.
Example 8-2 Launching the database configuration
sysmgr:/etc/TWGServer/ # passwd postgres
Changing password for postgres
142 The IBM Eserver BladeCenter JS20

New password:
Re-enter new password:
Password changed
sysmgr:/etc/TWGServer/ # cd /opt/IBM/director/bin/
sysmgr:/opt/IBM/director/bin # ./cfgdb
7. The installer launches. In the first window (Figure 8-1), select PostgreSQL.
Figure 8-1 Selecting the type of database
Chapter 8. System management scenarios 143

8. Enter the database connection information, as shown in Figure 8-2. Click
Next to complete the configuration of the database.
Figure 8-2 Database connection information
144 The IBM Eserver BladeCenter JS20

9. We decided to use encrypted communication for our IBM Director
environment. To do this, execute cfgsecurity as shown in Figure 8-3.
sysmgr:/opt/IBM/director/bin # cfgsecurity
Do you want IBM Director to encrypt data communications on this system?
0 - No
1 - Yes
Enter number corresponding to desired security setting (0 is default):
1
Which encryption algorithm do you want to use?
1 - DES encryption (56-bit key)
2 - Triple DES encryption (128-bit key)
2
0 - No encryption
Encryption set or unset successfully.
All IBM Director servers and encrypted systems are set to "secured"
automatically.
Setting secured or unsecured system successful.
Figure 8-3 Setting Director server encryption
10.After you configure the database and security, start the Director daemon.
sysmgr:/opt/IBM/director/bin # ./twgstart
With the daemon started, the Director server configuration is now complete.
8.1.2 Installing an IBM Director Agent
Follow these steps to install an IBM Director Agent.
1. Mount the Director CD and copy the files as explained in steps 1 and 2 on
page 142. This time, substitute /media/cdrom/director/server/linux/i386/ with
/media/cdrom/director/agent/linux/ppc/ in step 2.
Chapter 8. System management scenarios 145

2. With the sources copied to the hard drive, launch the installer, as shown in
Figure 8-4.
blade:/root/Director-Source/ # ./dirinstall
Attempting to install ITDAgent-4.12-1.ppc.rpm
Preparing...
#################################################
ITDAgent
#################################################
To start the IBM Director Agent manually, run /opt/ibm/director/bin/twgstart
Installation of selected components is successful.
To start the IBM Director Agent manually, run /opt/ibm/director/bin/twgstart
Figure 8-4 Installing the Director Agent
3. After the installation is complete, configure security as desired. We chose to
use encryption, as shown in Figure 8-5.
blade:/etc/TWGServer/ # cd /opt/IBM/director/bin/
blade:/opt/IBM/director/bin/ # cfgsecurity
Do you want IBM Director to encrypt data communications on this system?
0 - No
1 - Yes
Enter number corresponding to desired security setting (0 is default):
1
Encryption set or unset successfully.
All IBM Director servers and encrypted systems are set to "secured"
automatically.
Setting secured or unsecured system successful.
Figure 8-5 Configuring Director Agent security
4. With the security set, start the Director Agent.
blade:/opt/IBM/director/bin # ./twgstart
146 The IBM Eserver BladeCenter JS20

8.1.3 Using Director to manage JS20s
With the agent installed, you can add the BladeCenter JS20 to the Director
console as another managed server. For details about how to do this, see the
IBM Director 4.12 Installation and Configuration Guide, SC09-P290-50.
The following subset of Director management features are available for the
BladeCenter JS20:
► Software Distribution Standard Edition
► Software Distribution Premium Edition
► Update Assistant
► Inventory Hardware
► Inventory Software
► Event Action PLans
► Hardware Health Status
► Resource Monitors
► Eventlog
► Agent Discovery
► Process Management
► SNMP Alerts
► SNMP Browser
► Rack Manager
► Remote Session
► File Transfer
8.2 IBM Cluster Systems Management
BladeCenter JS20 installations may involve clusters of many blade servers. Many
of these installations use IBM Cluster Systems Management to both install and
manage BladeCenter JS20s.
CSM for Linux has been under development since early 2000 and was first
released as a product in June 2001. The original requirement for this product was
to provide common cluster management for Linux and AIX, leveraging IBM’s
existing cluster software portfolio to work with Open Source packages and best
practices from the Linux cluster market.
CSM provides basic cluster management functions, such as hardware control
and operating system and software installation. It also provides significant
value-added functions such as configuration file management, a robust
monitoring infrastructure, automated event management, diagnostic probes, and
common management of AIX and Linux clusters.
Chapter 8. System management scenarios 147

As CSM was being developed, the Extreme Cluster Administration Toolkit
(xCAT), a script-based package, was developed by IBM’s advanced technical
sales support team. It was provided to clients who purchase Linux clusters based
on IBM servers. It was also used by IBM Global Services to address their need
for tools to deploy and manage Linux clusters. In the interim, xCAT became a
proving ground for new concepts and practices centered on the management of
Linux clusters. It can be a model for collaborative development between IBM and
our clients.
At the time of the development of this book, CSM was available in its fourth
release (V1.4.0.4) with expanded scalability and hardware support. Many xCAT
functions have been integrated into CSM, along with utilities to automate the
process of migrating from xCAT to CSM. Through the use of CSM, clients who
have implemented xCAT can now have many of these functions with full product
support from IBM.
The key benefits of CSM can help to reduce costs, provide higher availability of
the cluster for productive use, and improve system utilization. Customers who
have existing AIX 5L operating system-based cluster systems can leverage those
skills to manage their Linux clusters.
System administrators can automate repetitive installation and configuration
tasks and automate problem determination and recovery. They can monitor and
report health information and resource utilization, and automatically recover from
node, storage, or network failures. This can lead to overall simplification of cluster
administration.
This section describes a fast-path CSM implementation to install and manage
BladeCenter JS20s. For a more comprehensive view, refer to the CSM for Linux
Planning and Installation Guide, SA22-7853.
Our implementation of CSM assumes the type of network configuration
described in 4.1.1, “Minimal network requirements” on page 54. Review this
along with other CSM planning considerations described in 4.3.2, “IBM Cluster
Systems Management” on page 64.
8.2.1 Setting up a CSM management node
The first step in installing CSM is to set up a management node. We focus on
how to set up a management node based on an xSeries server. We believe this
is a common scenario for users of BladeCenter JS20s, particularly if they also
have other blade servers that use Intel central processing units (CPUs).
Using an xSeries server to install and manage BladeCenter JS20s cluster nodes
introduces some complications that are not required when the management
148 The IBM Eserver BladeCenter JS20

node is the same architecture as the compute nodes. We focus our discussion
around these issues.
We use CSM to both install and manage BladeCenter JS20s. Therefore, we must
use a compatible operating system on both the management node and the
BladeCenter JS20 cluster nodes. The steps we require to set up the CSM
management node are:
1. Install a supported operating system.
2. Configure network interfaces and set up host name resolution.
3. Obtain the latest CSM software and prerequisites.
4. Install the CSM base RPM.
5. Run the CSM installation program.
6. Copy the operating system, service packs, CSM code, and prerequisites for
BladeCenter JS20 compute nodes.
7. Accept the CSM license.
8. Set up hardware control point passwords.
9. Verify the installation.
We review each of these steps in the sections that follow. For our tests, we
performed a standard installation of SUSE LINUX Enterprise Server (SLES) 9 for
x86 on our management node directly from the distribution CDs.
The management node network configuration
Our implementation of CSM assumes the type of network configuration
described in 4.1.1, “Minimal network requirements” on page 54.
The CSM management node is connected to both the hardware management
subnet and the operating system management subnet using separate network
interfaces. You can use additional network interfaces to connect the management
node to other networks if required.
In our scenario, we used the IP address ranges 9.3.5/24 for the hardware
management subnet and 192.168.1/24 for the operating system management
subnet. There should be no other active DHCP servers on the operating system
management subnet, because the CSM management node performs the DHCP
server function on that subnet.
Define host names and allocate IP addresses for the following interfaces before
you attempt to use CSM:
► The management node network interface that connects to the operating
system management subnet
Chapter 8. System management scenarios 149

This is the interface that will be used to install the operating system on the
BladeCenter JS20 cluster nodes.
► The network interface on each BladeCenter JS20 that connects to the
operating system management subnet
► The external network interface of each BladeCenter Management Module
You can either set up a hosts file on the management node or use a Domain
Name System (DNS) server to define the host name to IP address mapping. In
our scenario, we used a hosts file as shown in Example 8-3.
Example 8-3 /etc/hosts on management node
127.0.0.1 localhost
192.168.1.20 mgr
192.168.1.117 blade1
192.168.1.118 blade2
192.168.1.119 blade3
9.3.5.190 bcmm
Obtaining the latest CSM software and prerequisites
You can obtain the latest CSM software from:
http://techsupport.services.ibm.com/server/cluster/fixes/csmfixhome.html
Check this Web site for the latest CSM software even if you have CSM available
on CD, because new CSM software releases occur every few months.
The CSM software available at the previous Web site is under a 60-day Try and
Buy license. If you are performing a permanent CSM installation, you need to
have purchased an appropriate quantity of CSM licenses. You also need the full
license file that is contained on the CSM product CD.
We downloaded two different versions of CSM: one for the management node
(xSeries based) and one for the BladeCenter JS20 cluster nodes. The two
versions of CSM that we downloaded were:
► CSM for Linux on xSeries V1.4.0.4
► CSM for Linux on pSeries V1.4.0.4
Follow these steps:
1. Type the following command:
mkdir /tmp/csm_i386
2. Now type this command:
mkdir /tmp/csm_js20
150 The IBM Eserver BladeCenter JS20

3. Download the two CSM tarballs into their respective directories created in
steps 1 and 2.
4. Change to /tmp/csm_i386 and run the following command:
tar -xvzf csm-linux-1.4.0.0.i386.tar.gz
5. Change to /tmp/csm_js20 and run the following command:
tar -xvzf csm-linux-1.4.0.ppc64.tar.gz
6. We created a directory for such prerequisite packages as Autoupdate called
/tmp/csmpreq. Autoupdate is a package that is used by CSM. We
downloaded Version 5.3.11-1 from the following URL and placed it into the
/tmp/csmpreq directory.
http://ftp.mat.univie.ac.at/pub/teschl/autoupdate/index.html
Installing core CSM RPM
You must be logged into the management node as root to perform this step.
We installed the core CSM RPM on the management node using this command:
# rpm -ivh /tmp/csm_i386/csm.core*.i386.rpm
When you run the rpm command and install the CSM core, CSM sets the $PATH
and $MANPATH environment variables. Therefore, you must log out and then log
in again to the management node after performing the installation to accept the
changes to the environment variables that are required when performing
subsequent installation steps.
Running CSM installation program
After logging back into the management node as root, we ran the CSM
installation program using the following command:
# installms -p /tmp/csm_i386:/tmp/csmpreq
Note: The installms command is found in the /opt/csm/bin directory after you
install the CMS core.
The -p option of the installms command allows you to specify the location where
the CSM installation program should search for the files it needs. We specified
the two directories where we placed the CSM software and prerequisites for the
management node hardware architecture that we were using.
The installms command prompts you to insert operating system CDs if it cannot
find the operating system packages that it needs in one of the directories listed
after the -p option. You are also prompted to replace the Trivial File Transfer
Protocol (TFTP) daemon provided with the Linux distribution with a replacement
Chapter 8. System management scenarios 151

version packaged with CSM. And you are prompted to accept the licenses
associated with some software packages distributed with CSM.
Make sure you resolve any errors reported by the installms command before
you continue with the next step.
Copying the code required for BladeCenter JS20s
The BladeCenter JS20 has a different hardware architecture than the xSeries
server that we used for our management node. Therefore, you must copy the
code required for the BladeCenter JS20s to the management node. This
includes:
► SUSE LINUX Enterprise Server 9 for IBM POWER
► CSM for Linux on pSeries
► CSM prerequisites
We used the CSM copycds and copycsmpkgs commands to copy the operating
system, service pack, CSM software, and prerequisites for the BladeCenter JS20
cluster nodes to the management node. Both of these commands require you to
specify any attributes that are different to the operating system environment on
the management node. In our scenario, we specified the InstallPkgArchitecture
attribute, because the blade server cluster nodes have a different architecture
than the management node.
We copied the operating system CDs using the following command:
# copycds InstallPkgArchitecture=ppc64
We copied the CSM software and prerequisites for the BladeCenter JS20 cluster
nodes using this command:
# copycsmpkgs -p /tmp/csm_js20:/tmp/csmpreq InstallPkgArchitecture=ppc64
The copycsmpkgs command prompts you to insert operating system CDs if it
cannot find the operating system packages that it needs in one of the directories
listed after the -p option.
Accepting the CSM licence
Accept the CSM licence. If you want to use a 60-day trial licence, use the
following command:
# csmconfig -L
If you want to accept the permanent license, mount the CSM product CD and run
the following command:
# csmconfig -L /media/cdrom/csmlum.full
152 The IBM Eserver BladeCenter JS20

Setting up hardware control point passwords
CSM requires user IDs and passwords to communicate with BladeCenter
Management Modules so that it can:
► Control power to BladeCenter JS20s
► Provide remote console access to BladeCenter JS20s
► Retrieve the MAC addresses for the Gigabit Ethernet interfaces on
BladeCenter JS20s
CSM lets you define one user ID and password for power control and a different
user ID and password for remote console access for each BladeCenter
Management Module. Both of these profiles must match login profiles that are
defined in the corresponding BladeCenter Management Module. You can
configure these profiles through the BladeCenter Management Module Web
interface.
In a production environment, you should define two different login profiles, one
for power control and the other for remote console access. You should also limit
the access for each of these profiles to the minimum needed. In our test
environment, we used the same profile for both power control and remote
console access.
We recommend that you use the same user ID and password for power control
for all BladeCenter Management Modules in a cluster and define these as the
default user ID and password for power control in CSM.
You define a default BladeCenter Management Module user ID and password for
power control using the following command:
# systemid -p blade USERID
The systemid command prompts you to enter the password associated with the
default user ID you specified.
Similarly, we recommend that you use the same user ID and password for remote
console access for all BladeCenter Management Modules in a cluster and define
these as the default user ID and password for remote console access in CSM.
You define a default BladeCenter Management Module user ID and password for
remote console access using the following command:
# systemid -c -p blade USERID
The systemid command prompts you to enter the password associated with the
default user ID you specified. This step is necessary for the Serial over LAN
(SoL) access.
Chapter 8. System management scenarios 153

You can verify what user IDs you configured as the default for BladeCenter power
control and remote console access by issuing the systemid command with no
parameters. This displays output similar to what is shown in Example 8-4.
Example 8-4 Output from systemid command
blade_#DEFAULT# USERID
blade_#DEFAULT#_console USERID
Verifying the installation
CSM is provided with an installation verification probe that can pick up many
installation errors. You can run this probe by using the following command:
# probemgr -p ibm.csm.ms -l 0
Review the output of this command and investigate any error messages.
8.2.2 Installing and managing BladeCenter JS20s using CSM
After you establish a CSM management node, you can install and manage
BladeCenter JS20s.
The required steps are:
1. Define your BladeCenter JS20 cluster nodes to CSM.
2. Set up the installation environment.
3. Start installation and wait for completion.
We examine each of these steps in more detail in the sections that follow.
Defining the BladeCenter JS20 cluster nodes
We find that the easiest way to define BladeCenter JS20 cluster nodes to CSM is
to use a node definition file. There are many other ways to define nodes in CSM.
Review the CSM for Linux Planning and Installation Guide, SA22-7853, to
determine the most appropriate method for your environment.
The node definition file that we used for our small cluster of BladeCenter JS20s is
shown in Example 8-5.
Example 8-5 CSM node definition file
default:
ManagementServer=mgr
PowerMethod=blade
HWControlPoint=bcmm
ConsoleMethod=blade
ConsoleServerName=bcmm
154 The IBM Eserver BladeCenter JS20

lade1:
blade2:
blade3:
InstallDiskType=ide
InstallPkgArchitecture=ppc64
InstallAdapterName=eth1
HWControlNodeId=blade1
ConsolePortNum=1
HWControlNodeId=blade2
ConsolePortNum=2
HWControlNodeId=blade3
ConsolePortNum=3
The default stanza of the node definition file contains the node attributes that are
common to all the BladeCenter JS20 cluster nodes in our cluster. You can have
multiple default stanzas in a node definition file, with the new defaults applying to
those stanzas listed in the node definition file after the new default stanza.
The remaining stanzas define each BladeCenter JS20 cluster node, and contain
only those node attributes that are unique to that cluster node. The name of the
stanza for each BladeCenter JS20 cluster node should match the host name that
you assigned to that cluster node in your hosts file or DNS.
Use the following guide when you specify attribute values:
► ManagementServer: Host name of management node network interface
connected to the operating system management subnet
► HWControlPoint: Host name of the BladeCenter Management Module
► ConsoleServerName: Host name of the BladeCenter Management Module
► InstallAdapterName: Set to eth1 if you are using the network configuration
described in 4.1.1, “Minimal network requirements” on page 54
This network configuration performs operating system network installation via
eth1 to prevent issues with the SoL remote text console function, which uses
eth0.
► HWControlNodeId: Same name you assigned the BladeCenter JS20 cluster
node when you set up the hardware
We recommend that you make it the same as the host name. We demonstrate
how to configure this in 5.3, “Blade server configuration” on page 76.
► ConsolePortNum: Bay in the BladeCenter chassis where the BladeCenter
JS20 cluster node is installed
Chapter 8. System management scenarios 155

► PowerMethod: Should always be blade for BladeCenter JS20 cluster nodes
► ConsoleMethod: Should always be blade for BladeCenter JS20 cluster
nodes
After you create a node definition file for your cluster, you can define the nodes to
CSM using the following command:
# definenode -f /tmp/nodedefs
After the definenode command runs, the management server is set up with all
the node information for CSM, and you are ready to verify the node definitions.
Since the actual node installation has not happened yet, you can make changes
to any of the node definitions at this time. To determine whether the nodes have
been defined, enter the lsnode command from the management server.
After you run the lsnode command, the system responds with a line for each
node that was successfully defined. If a node is not defined, it does not appear in
the output for the lsnode command.
To display all the information about each node, use the lsnode command from
the management server with the –l (lowercase L) option, such as:
# lsnode -l
The system responds with a list (output) that contains the extended information
for each node that was successfully defined. If a node is not defined, it does not
appear in the output for the lsnode command.
Some of the attributes for a node may have null values at this point. If a node is
not defined, it is not installed. If you must correct the information, you can change
the attributes of the nodes either by running the chnode command, or by
rerunning the definenode command with the -m (modify) flag. The definenode -m
command accepts a new nodedef file or a new command line and only modifies
nodes that have changed attributes. To change attribute values of a node, enter
the chnode command from the management server:
chnode hostname attr=value
Setting up the installation environment
After you successfully define the node, configure how the node is to be installed
in the cluster. For Red Hat AS or Red Hat EL 3 installations, use Kickstart, and
run the csmsetupks command. For SUSE LINUX Enterprise Server installations,
use AutoYaST, and run the csmsetupyast command. The csmsetupks and
csmsetupyast commands configure the install process, collect MAC addresses,
and configure the network boot of the nodes.
156 The IBM Eserver BladeCenter JS20

In our lab environment using SuSE, the csmsetupyast did several things,
including:
► Retrieved the MAC addresses for the Gigabit Ethernet interfaces of each
BladeCenter JS20 cluster node
► Built a Autoyast configuration file for each BladeCenter JS20 cluster node
from a template
► Built the DHCP server configuration file /etc/dhcpd.conf
Before you attempt to use csmsetupyast for the first time, we recommend that
you verify that power control and remote console functionality is working
correctly. You can verify that power control is working using the following
command:
# rpower -a query
This command displays the current power status of all nodes in the cluster. If this
is not successful, determine why before you proceed any further.
You can verify that the remote consoles are working by using the following
command:
# rconsole -a
This command opens a separate console window for every cluster node that is
defined to CSM. If a BladeCenter JS20 cluster node is not powered on, you see
messages indicating that the SoL is not ready. Close each remote console
window using the key sequence Ctrl+E c . before you continue further.
When you are satisfied that power control and remote console functionality is
working correctly, verify that the DHCP server on the management node is
configured to listen on the correct network interface.
Under SUSE LINUX Enterprise Server, the interfaces that the DHCP server uses
to listen are configured in the file /etc/sysconfig/dhcpd. Set the
DHCP_INTERFACE variable in this file to include the Ethernet interface on the
management node that is connected to the operating system management
subnet described in “Operating system management subnet” on page 57.
Now set up the installation environment using the CSM command:
# csmsetupyast -x -P
We use the -x option because we already copied the necessary files to support
the network installation of the operating system to the management node. If you
do not specify this option, you are prompted to insert operating system CDs. We
use the -P option to specify all nodes that are in the premanaged state and are
waiting to be installed.
Chapter 8. System management scenarios 157

The csmsetupyast command uses an XML template file found in /opt/csm/install
as a default template for the Autoyast configuration files that it creates for each
BladeCenter JS20 cluster node. Depending upon the release of SuSE that you
are using at the time, the file has the format:
yastcfg.InstallDistributionNameInstallDistributionVersion-Arch.xml
For example, in our lab environment, the file was called
yastcfg.SLES9-ppc64.xml.
If you want to use a different template, first create the template and then specify it
on the csmsetupyast command using the -k option. We recommend that you
copy the default template when creating new templates to ensure that you
include all the elements that CSM requires in the template.
Review the output of the csmsetupyast command carefully and resolve any errors
before you proceed further.
Starting the cluster node installation
Start the cluster node installation using the installnode command. We
recommend that you try to install a few nodes at first and make sure it works
before you try to install many the nodes concurrently. To start installation of a
single node, use the command:
installnode -n blade1
The installnode command takes a few minutes to complete when used with
BladeCenter JS20 cluster nodes. This is because the installnode command
must wait until the BladeCenter JS20 boot process has reached the point where
it can pass network installation parameters to the firmware.
When the installation is underway, the installnode command exits and displays
the status for each node it was asked to install. You can monitor the status of the
install as it progresses using the monitorinstall command. You can also open a
remote console to watch the installation progress using the rconsole command.
For more information about installing CSM, see CSM for Linux: Planning and
Installation Guide, SA22-7853.
158 The IBM Eserver BladeCenter JS20

Abbreviations and acronyms
ac alternating current
AIX Advanced Interactive
eXecutive
BIU Bus Interface Unit
BOOTP Bootstrap Protocol
BOS base operating system
BPU branch processing unit
BSMP Blade System Management
Processor
CISC complex instruction set
computer
CLI command line interface
CRLF carriage return/line feed
CSM Cluster Systems
Management
CTR count register
DC direct current
DDR double data rate
DHCP Dynamic Host Configuration
Protocol
DNS Domain Name Server
ECC Error Correction Code
ERAT effective to real address
translation
ESM Ethernet Switch Module
FPR Floating Point Register
FPU Floating Point Unit
FSB Front Side Bus
FTP File Transfer Protocol
FXU Fixed Point Unit
Gb gigabit
GB gigabyte
gcc GNU C Compiler
GPR General Purpose Register
GUI Graphical User Interface
HTTP Hypertext Transfer Protocol
HTTPS Hypertext Transfer
Protocol-Secure
I/O input/output
IBM International Business
Machines Corporation
IGESM CISCO Intelligent Gigabit
Ethernet Switch Module
IP Internet Protocol
IPL Initial Program Load (Boot)
ITSO International Technical
Support Organization
KVM keyboard, video and mouse
LAN local area network
LPAR logical partition
LR Link Register
MAC Media Access
Mb megabit
MB megabyte
MFLOPS Million Floating Point
Operations Per Second
MP multiprocessor
NFS Network File System
NIC Network Interface Controller
NIM Network Install Manager
PKT packet
POSIX Portable Operating System for
UNIX
POST Power On Self Test
POWER Performance Optimization
With Enhanced RISC
RAM random access memory
© Copyright IBM Corp. 2005. All rights reserved. 159

RHEL Red Hat Enterprise Linux
RISC Reduced Instruction Set
Computer
ROM Read Only Memory
RTC Real Time Clock
SAN storage area network
SCSI small computer systems
interface
SMP Simultaneous Multi-Processor
SOL Serial over LAN
SPOT Shared Product Object Tree
SSH Secure Shell
TCP/IP Transmission Control
Protocol/Internet Protocol
TFTP Trivial File Transfer Protocol
TLB Translation Lookaside Buffer
USB Universal Serial Bus
UXBC UpdateXpress firmware
update scripts for
BladeCenter
VLAN virtual local area network
VMX Vector Multimedia eXtensions
VPN Virtual Private Network
VXU Vector Processing Unit
WAN Wide Area Network
XER Fixed Point Exception
Register
XML eXtended Markup Language
YAST Yet Another Setup Tool
(SuSE)
160 The IBM Eserver BladeCenter JS20

Related publications
IBM Redbooks
The publications listed in this section are considered particularly suitable for a
more detailed discussion of the topics covered in this redbook.
For information about ordering these publications, see “How to get IBM
Redbooks” on page 164. Note that some of the documents referenced here may
be available in softcopy only.
► Implementing System Management Solutions using IBM Director, SG24-6188
► The Cutting Edge: IBM Eserver BladeCenter, REDP-3581
► IBM Eserver BladeCenter Systems Management, REDP-3582
► Deploying Lotus Domino on IBM Eserver BladeCenter, REDP-3584
► Deploying Apache on IBM Eserver BladeCenter, REDP-3588
► Deploying Samba on IBM Eserver BladeCenter, REDP-3595
► IBM Eserver BladeCenter Networking Options, REDP-3660
► IBM Eserver BladeCenter Layer 2-7 Network Switching, REDP-3755
► IBM Eserver BladeCenter Systems Management with IBM Director V4.1
and Remote Deployment Manager 4.1, REDP-3776
► IBM Eserver BladeCenter Configuration Tips, TIPS-0454
Other publications
These publications are also relevant as further information sources:
► Peterson and Davie, Computer Networks: A Systems Approach, 3rd Edition,
Morgan Kaufmann, 2003, ISBN 1-55-860832-3
► Stevens, TCP/IP Illustrated, Volume 1: The Protocols, Addison-Wesley, 1993,
ISBN 0-20-163346-9
► Tanenbaum, Computer Networks, 4th Edition, Prentice Hall, 2002, ISBN
0-13-066102-3
► Tanenbaum, Modern Operating Systems, 2nd Edition, Prentice Hall, 2001,
ISBN 0-13-031358-0
© Copyright IBM Corp. 2005. All rights reserved. 161

Online resources
► CSM for Linux: Planning and Installation Guide, SA22-7853
http://publib.boulder.ibm.com/epubs/pdf/am7il103.pdf
► IBM Cluster Systems Management for Linux Planning and Installation Guide,
SA22-7873
► IBM Cluster Systems Management for AIX5L Planning and Installation Guide,
SA22-7919
► IBM Director 4.12 Installation and Configuration Guide, SC09- P290-50
► IBM Director 4.0 for BladeCenter Products: Systems Management Guide,
SC01-R051-40
► CSM for Linux Planning and Installation Guide, SA22-7853
These Web sites and URLs are also relevant as further information sources:
► Installing AIX 5L on the IBM Sserver BladeCenter JS20 whitepaper
http://www.ibm.com/servers/aix/whitepapers/aix_js20.pdf
► IBM Eserver Support Web site
http://www.ibm.com/pc/support
► Red Hat Linux
http://www.redhat.com
► SUSE LINUX
http://www.suse.com
► Instructions for installing SuSE from CD
http://www.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm&lndocid=
MIGR-54819
► PowerPC 970 Training
http://www.technonics.com/powerpc.htm
► Linux at IBM
http://www-1.ibm.com/linux
► Linux Kernel Archives
http://www.kernel.org
► AIX Version 5L Operating System
http://www-1.ibm.com/servers/aix
162 The IBM Eserver BladeCenter JS20

► IBM Microelectronics Division - PowerPC Technical Documentation
http://www.chips.ibm.com
► IBM ^ BladeCenter Power Module Upgrade Guidelines Technical
Update
http://www.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-53353
► Myrinet Web site
http://www.myri.com
► IBM ^ Cluster 1350 product
http://www.ibm.com/eserver/cluster
► PowerPC Architecture books from the IBM DeveloperWorks Web site
http://www.ibm.com/developerworks/eserver/articles/archguide.html
► IBM white paper POWER3: Next Generation 64-bit PowerPC Processor
Design
http://www.ibm.com/servers/eserver/pseries/hardware/whitepapers/power3wp.ht
ml
► IBM white paper entitled POWER4 System Microarchitecture
http://www.ibm.com/servers/eserver/pseries/hardware/whitepapers/power4.html
► PowerPC Microprocessor Family: AltiVec Technology Programming
Environments Manual
http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/FBFA164F824370F98
7256D6A006F424D
► RETAIN tip H181655
http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-55282
► CSM publication site
http://publib.boulder.ibm.com/clresctr/windows/public/clusterbooks.html
► Service and productivity tools for Linux on POWER
http://techsupport.services.ibm.com/server/lopdiags
► The Linux Documentation Project - Kernel HOWTO
http://www.tldp.org
► The latest CSM software
http://techsupport.services.ibm.com/server/cluster/fixes/csmfixhome.html
► CSM for Linux: Planning and Installation Guide, SA22-7853
http://publib.boulder.ibm.com/epubs/pdf/am7il103.pdf
Related publications 163

How to get IBM Redbooks
Help from IBM
You can search for, view, or download Redbooks, Redpapers, Hints and Tips,
draft publications and Additional materials, as well as order hardcopy Redbooks
or CD-ROMs, at this Web site:
ibm.com/redbooks
IBM Support and downloads
ibm.com/support
IBM Global Services
ibm.com/services
164 The IBM Eserver BladeCenter JS20

Index
Symbols
/boot/ppc64 109
/etc/dhcpd.conf 110
/etc/exports 103
/tftpboot 111
Numerics
64-bit computing 32, 34, 36–37, 42, 46
A
activating the open firmware interface 94
Aggressive branch prediction 44–45
AIX operating system 48, 54, 57, 60, 62, 64, 79,
84–85
installing 123
installing with NIM 124–138
Altivec 40
AMD 3DNow! 40
Apple Computer 36
ARP function 56
array 39
array processing 39
AS/400 37
AutoYaST 114
B
blade servers 21
assigning names 77
configuration 76–78
setting the boot sequence 78
Blade Systems Management Processor (BSMP)
15, 22, 26
BladeCenter
advantages
high availability 4
lower cost 4
SAN optimization 4
server consolidation 4
switch technology 4
chassis 8–10
management module 13
HS20 blade server 21
I/O modules 16–21
Management Module 13–16
power module options 10
power options 10–163
power requirements 5
standard features 9
supported bays 8
versus IBM rack server 5
BladeCenter T 10
boot sequence 78
Bootstrap protocol (BOOTP) 60
Brocade Enterprise SAN Switch Module 17, 19
Brocade Entry SAN Switch Module 17, 20
Bus Interface Unit (BIU) 45
C
C2 security 48
cache structure 44
Cisco Systems Intelligent Gigabit Ethernet Switch
Module 17–18, 76, 82
Command line interface (CLI) 56, 91
env command 92
Complex instruction set computer (CISC) 30
console for JS20 blades 87
copycds command 152
copycsmpkgs command 152
count register (CTR) 35
CSM
see IBM Cluster Systems Management
csmconfig command 152
D
definenode command 156
DHCP (dynamic host configuration protocol) 60–61
dynamic host configuration protocol (DHCP) 54,
60–61, 70, 110, 149
E
effective to real address translation (ERAT) 44
en0 62
en1 62
env command 92
© Copyright IBM Corp. 2005. All rights reserved. 165

eth0 62
eth1 62
F
Fibre Channel Expansion Card 28
firmware 79–87
JS20 blade server 84
LAN switch I/O module 82
Management Module 80
upgrading under Linux 84
vital product data (VPD) 79
fixed-point exception register (XER) 34
floating-point status and control register (FPSCR)
34
G
general purpose registers (GPRs) 34
Gigabit Ethernet Expansion Card 28
GNU C Compiler (gcc) 49
H
high availability 4
HTTP 16
HTTPS 16
I
I/O modules 16–21, 73–75
configuration 73
IBM 4-Port Gigabit Ethernet Switch Module 17, 76,
79, 82
IBM AS/400 37
IBM Cluster Systems Management (CSM) 16,
50–51, 56, 60, 64, 66, 147–158
obtaining latest software and prerequisites 150
setting up 148
IBM Director 16, 56, 63–64, 140
agent 64
installing agent 145
IBM General Parallel File System (GPFS) 59
IBM Microelectronics Division 46
chip documentation 46
IBM Microprocessors
POWER2 31
POWER3 37
POWER4 37
POWER5 38
PowerPC 970 22, 29, 38, 42, 46
166 The IBM Eserver BladeCenter JS20
PowerPC 970FX 22, 29, 38, 42, 46
IBM microprocessors
PowerPC 601 36
PowerPC 603 36
PowerPC 604 36
IBM rack server 5
IBM RS/6000 36
installms command 151
installnode command 158
integrated switch technology 4
Integrated Systems Management Processor (ISMP)
85
Intel Multimedia Extensions (MMX) 40
Intel Streaming SIMD Extensions (SSE) 40
J
JS20 Blade Server
base features 24
ethernet controller 25
I/O expansion option connectors 25
I/O subsystem 24
IDE controller 25
memory subsystem 24
miscellaneous I/O 26
optional features 27
expansion cards 28
hard disk drives 27
memory modules 27
processor subsystem 24
USB controllers 26
K
keyboard, video monitor and mouse (KVM) 14
KVM (keyboard, video monitor and mouse) 14
L
link register (LR) 35
Linux
compiling a Red Hat kernel 108
compiling a SuSe kernel 108
configuring the sources 102
installing 102
Red Hat sources 102
SuSE sources 104
local area network (LAN) 16
lsnode command 156

M
Management Module 13–16, 70–73
configuration 70–73
console commands 93
default IP address 70
default password 72
default userid 71
firmware 80
mkinitrd command 109
mkzimage_cmdline command 117
Motorola 36
Myrinet 16, 20, 22, 28, 58–59
N
Network File System (NFS) 60
network installation 118
network installation planning 60
network interface selection 62
network planning 54
minimal requirements 54
network services 60
Nortel Networks Layer 2-7 Gigabit Ethernet Switch
Module 17–18, 76, 82
O
obtaining latest CSM software and prerequisites
150
Open Firmware 117
Operating Environment Architecture (OEA) 33
operating system support 48
Optical Pass-thru Module 20
P
PCI
see Peripheral Component Interconnect
performing a network installation 118
performing an unattended installation 116
Peripheral Component Interconnect (PCI) 5, 25,
141
POWER architecture
definition 30
POWER2 31
POWER3 37
POWER4 37
POWER5 38
Power On Self Test (POST) 109
power requirements for BladeCenter 5
PowerPC architecture
background 32
definition 30
Operating Environment Architecture (OEA) 33
PowerPC 601 36
PowerPC 603 36
PowerPC 604 36
PowerPC 970 22, 29, 38, 42, 46
PowerPC 970FX 22, 29, 38, 42, 46
User Instruction Set Architecture (UISA) 33
Virtual Environment Architecture (VEA) 33
preparing an unattended installation 111
probemgr command 154
R
rack server 5
rconsole command 157
Red Hat Enterprise Linux 3 48, 121
Red Hat’s Kickstart 112
Redbooks Web site 164
Contact us xv
reduced instruction set computer (RISC) 30
Register renaming 44
required network services 60
rpm command 151
rpower command 157
RS64 processors 36
rtas_flash kernel module 84
S
segment lookaside buffer (SLB) 44
Serial over LAN (SoL) 14, 50, 57, 62, 76, 87
components 88
configuring 89
using command line interface 91
server consolidation 4
setting up a CSM management node 148
setting up network installation 61
small computer system interface (SCSI) 5
SoL (Serial over LAN) 14
SSH 16, 89, 91
storage area network (SAN) 16
optimization 4
superscalar features 44
SuSE AutoYaST 114
SUSE LINUX Enterprise Server 9 49, 118
symmetrical multiprocessor (SMP) 36
systemid command 153
Index 167

T
translation lookaside buffer (TLB) 44
Trivial File Transfer Protocol (TFTP) 60, 111
U
unattended installation 111, 116
Universal Manageability Initiative 49
update_flash script 85
User Instruction Set Architecture (UISA) 33
V
vector 39
vector length 38
vector register file (VRF) 41
Vector/SIMD Multimedia eXtension (VMX) 38
Virtual Environment Architecture (VEA) 33
Virtual LAN (VLAN) 15, 17–18, 54, 57, 61, 76, 87,
89
vital product data (VPD) 79
VMX extensions to PowerPC architecture 40
VMX status and control register (VSCR) 41
VPD (vital product data) 79
VRP (vector register file) 41
VSCR (VMX status and control register) 41
X
xinetd daemon 111
Y
Yet Another Setup Tool (YaST) 104
Z
zImage.initrd 116
zImage.initrd file 107, 109
building your own 107–108
168 The IBM Eserver BladeCenter JS20

The IBM Eserver BladeCenter JS20

The IBM Eserver
BladeCenter JS20
Learn about IBM
Eserver
BladeCenter JS20
and blade server
technology
Understand the
PowerPC 970
microprocessor
architecture
Plan for, install, and
configure
BladeCenter JS20
SG24-6342-01 ISBN 073849061X
Back cover
Blade servers are a relatively new technology. They have
captured industry focus because of their modular design,
which can reduce cost with a more efficient use of valuable
floor space. They offer simplified management, which can
help to speed such tasks as installing, reprovisioning,
updating, and troubleshooting hundreds of blade servers. You
can do all of this remotely using one graphical console with
IBM Director systems management tools.
In addition, blade servers provide improved performance by
doubling current rack density. By integrating resources and
sharing key components, costs decrease and availability
increases.
The IBM Eserver BladeCenter boasts innovative modular
technology, leadership density, and availability. It was
designed to help solve a multitude of real-world problems.
This IBM Redbook takes an in-depth look at the IBM
Eserver BladeCenter JS20. This is a two-way blade server
for applications requiring 64-bit computing. It is ideal for
computer-intensive applications and transactional Internet
servers. This IBM Redbook helps you to install, tailor, and
configure the IBM Eserver® BladeCenter JS20.
INTERNATIONAL
TECHNICAL
SUPPORT
ORGANIZATION
®
BUILDING TECHNICAL
INFORMATION BASED ON
PRACTICAL EXPERIENCE
IBM Redbooks are developed by
the IBM International Technical
Support Organization. Experts
from IBM, Customers and
Partners from around the world
create timely technical
information based on realistic
scenarios. Specific
recommendations are provided
to help you implement IT
solutions more effectively in
your environment.
For more information:
ibm.com/redbooks