Configuration and Tuning GPFS for Digital Media ... - IBM Redbooks

More documents

Recommendations

Info

2.2.8 Quorum rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 2.2.9 GPFS memory management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 2.2.10 GPFS cluster models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 2.3 Designing your cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 2.3.1 Knowing your environment — questions to ask . . . . . . . . . . . . . . . . 69 2.3.2 GPFS considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 2.3.3 Storage design considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 2.3.4 Network and miscellaneous design considerations . . . . . . . . . . . . . 72 2.3.5 Classification of GPFS commands . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Chapter 3. Infrastructure configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 3.1 General considerations for installing GPFS . . . . . . . . . . . . . . . . . . . . . . . 82 3.2 Hardware setup and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.2.1 List of components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.2.2 Disk storage devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 3.2.3 Storage Area Network (SAN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 3.2.4 Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 3.2.5 Network infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 3.2.6 ITSO lab environment overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 3.3 Software installation (Linux-based cluster) . . . . . . . . . . . . . . . . . . . . . . . 103 3.3.1 Base operating system setup and configuration . . . . . . . . . . . . . . . 104 3.3.2 Linux storage device multipath configuration . . . . . . . . . . . . . . . . . 110 3.3.3 Basic network configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 3.3.4 Configuring secure shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 3.3.5 Time synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 3.4 GPFS installation and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 3.4.1 Install GPFS packages and define a GPFS cluster . . . . . . . . . . . . 127 3.4.2 Start GPFS and verify all nodes join the cluster . . . . . . . . . . . . . . . 134 3.4.3 Create NSDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 3.4.4 Create the file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 3.4.5 Mount the file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 3.4.6 Checking the cluster and file system configuration . . . . . . . . . . . . . 136 Chapter 4. Advanced topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 4.1 Linux multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 4.1.1 Using a vendor specific HBA multipath driver . . . . . . . . . . . . . . . . . 142 4.1.2 Using RDAC multipath driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 4.1.3 Using device mapper multipath driver with GPFS. . . . . . . . . . . . . . 147 4.2 SSH key distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 4.3 Samba server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 4.4 NFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 4.4.1 Installing NFS server on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 4.4.2 Configuring NFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 iv Configuration and Tuning GPFS for Digital Media Environments
4.4.3 Starting NFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 4.4.4 NFS in a digital media environment . . . . . . . . . . . . . . . . . . . . . . . . 160 4.4.5 Integrating NFS and GPFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 4.4.6 Preparing windows clients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 4.5 Additional GPFS management tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 4.5.1 Building gpfsperf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 4.5.2 Removing GPFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Chapter 5. Maintaining and tuning the GPFS environment. . . . . . . . . . . 171 5.1 Performance tuning cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 5.2 Tuning the GPFS environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 5.2.1 Tuning storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 5.2.2 Tuning the SAN hardware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 5.2.3 Tuning the operating system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 5.2.4 Tuning GPFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 5.2.5 Tuning the clients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 5.3 Software tools and utilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 5.3.1 System administration tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 5.3.2 System resource monitoring tools (Linux). . . . . . . . . . . . . . . . . . . . 199 5.3.3 Load generating tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 5.4 Client application considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 5.4.1 Client side buffering and failover mechanism . . . . . . . . . . . . . . . . . 206 5.4.2 Client side load balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 5.4.3 Block size considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 5.4.4 Mixed-media processing and file access patterns . . . . . . . . . . . . . 207 5.5 Performance tuning considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 5.6 List of GPFS related files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Appendix A. Basic infrastructure configuration additions . . . . . . . . . . . 211 A.1 Automatic network installation using SLES 9 Installation Server . . . . . . 212 A.2 Configuring tftpd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 Configuring tftpd for booting Intel-based computers (PXE). . . . . . . . . . . . 212 A.2.1 Configuring tftpd for booting POWER-based computers . . . . . . . . 213 A.3 Configuring dhcpd. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 A.4 SLES9 Installation Server Configuration with YaST using FTP . . . . . . . 215 A.5 Configuration of secure shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 A.6 Setting up storage software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 A.7 Installing and configuring a distributed shell . . . . . . . . . . . . . . . . . . . . . . 223 A.8 SLES9 Installation on an IBM eServer BladeCenter JS20 . . . . . . . . . . . 224 A.9 Sample storage profile file for DS4500 . . . . . . . . . . . . . . . . . . . . . . . . . . 225 Appendix B. Typical broadcast system workflow . . . . . . . . . . . . . . . . . . 231 Appendix C. Yam: Installation server setup tool . . . . . . . . . . . . . . . . . . . 239 Contents v
Page 1: Configuration and Tuning GPFS for D
Page 4 and 5: Note: Before using this information
Page 8 and 9: Abbreviations and acronyms . . . .
Page 10 and 11: Trademarks The following terms are
Page 12 and 13: Dag Wieers is a Linux and Open Sour
Page 14 and 15: Comments welcome Your comments are
Page 16 and 17: xiv Configuration and Tuning GPFS f
Page 18 and 19: 1.1 What is digital media? Digital
Page 20 and 21: Create Manage Distribute Transact F
Page 22 and 23: investments. And integration of dig
Page 24 and 25: 1.3.3 Broadcasting Table 1-2 Video
Page 26 and 27: 1.4 Broadcasting Within IBM this is
Page 28 and 29: 2. By using standard IT based compo
Page 30 and 31: Note: We are using the term offerin
Page 32 and 33: ► Streaming - Found in Digital Me
Page 34 and 35: These issues lead to the following
Page 36 and 37: ► Library Manager (LM) The LM pro
Page 38 and 39: ADMIRA does not provide the complet
Page 40 and 41: Storage-Layer Work-Layer Communicat
Page 42 and 43: ► Maintenance without downtime IB
Page 44 and 45: LDAP The OpenLDAP service installed
Page 46 and 47: 30 Configuration and Tuning GPFS fo
Page 48 and 49: 2.1 GPFS history and evolution The
Page 50 and 51: 2.1.1 Why choose GPFS? This section
Page 52 and 53: RPC services Other Nodes Configurat
Page 54 and 55: 2.2.3 GPFS block allocation GPFS us
Page 56 and 57:
a file (read or write) if it holds
Page 58 and 59:
Token Token received received for f
Page 60 and 61:
► minMissedPingTimeout Minimum am
Page 62 and 63:
lease missed lease renewal CfgMgr d
Page 64 and 65:
Note: Whenever possible, we recomme
Page 66 and 67:
4 Request for accessing a file 1 no
Page 68 and 69:
Data LUN0 In order to assure file a
Page 70 and 71:
Data LUN0 2.2.8 Quorum rules 2 3’
Page 72 and 73:
File system descriptor For maintain
Page 74 and 75:
General Parallel File System Failur
Page 76 and 77:
allocated at daemon start-up. The p
Page 78 and 79:
Figure 2-19 GPFS prefetch 2.2.10 GP
Page 80 and 81:
choices. Option two is using NSD on
Page 82 and 83:
Application Node With GPFS Daemon A
Page 84 and 85:
High SPEED IP INTERCONNECT GbE, 10G
Page 86 and 87:
► Application (compatibility, blo
Page 88 and 89:
- DS4500: At the time this book was
Page 90 and 91:
command node influencing / node rel
Page 92 and 93:
command node influencing / node rel
Page 94 and 95:
performance impacting commands mmrp
Page 96 and 97:
80 Configuration and Tuning GPFS fo
Page 98 and 99:
3.1 General considerations for inst
Page 100 and 101:
System component # Type Feature Hos
Page 102 and 103:
2109-F32 (switch1) 2109-F32 (switch
Page 104 and 105:
Figure 3-4 DS4500 - Save the storag
Page 106 and 107:
► Configuration for data LUNs: -
Page 108 and 109:
Note: For performance and availabil
Page 110 and 111:
GPFS_metadata2 1 B Host Group GPFS-
Page 112 and 113:
Figure 3-8 DS4500 Storage Manager G
Page 114 and 115:
3.2.4 Servers zoning configuration
Page 116 and 117:
Figure 3-10 LPAR profile of p5 550
Page 118 and 119:
The second network will be used as
Page 120 and 121:
Once the Linux operating system has
Page 122 and 123:
Figure 3-14 SLES9-yast-setup - Inst
Page 124 and 125:
Figure 3-16 Final installation conf
Page 126 and 127:
For example, if storage (LUNs, mask
Page 128 and 129:
Important: If RDAC/MPP is used, a n
Page 130 and 131:
INITRD_MODULES="ipr lpfcdd" where
Page 132 and 133:
An easy way to verify that the crea
Page 134 and 135:
proc/mpp/DS4500_1/controllerA: tota
Page 136 and 137:
Type: Direct-Access ANSI SCSI revis
Page 138 and 139:
problem with the GPFS cluster if DN
Page 140 and 141:
3.3.5 Time synchronization Check th
Page 142 and 143:
Windows client time synchronization
Page 144 and 145:
► Save the changes to site.mcr fi
Page 146 and 147:
* Descriptions and downloads for th
Page 148 and 149:
The disk descriptor file contains a
Page 150 and 151:
The pagepool parameter changes the
Page 152 and 153:
Example 3-25 Output of mount comman
Page 154 and 155:
138 Configuration and Tuning GPFS f
Page 156 and 157:
4.1 Linux multipathing The term mul
Page 158 and 159:
The Figure 4-3 shows the RDAC drive
Page 160 and 161:
Note: We recommend that you save a
Page 162 and 163:
► Make sure the mppUpper module i
Page 164 and 165:
Attention: We have tested the devic
Page 166 and 167:
kernel 2.6 user space kernel space
Page 168 and 169:
The order of the device maps indica
Page 170 and 171:
devices { device { vendor "IBM " pr
Page 172 and 173:
use of classic remote shell (rsh),
Page 174 and 175:
4.4 NFS Windows network protocol th
Page 176 and 177:
If everything loaded correctly, you
Page 178 and 179:
Forcing the NFS version When mounti
Page 180 and 181:
First prevent the NFS server from s
Page 182 and 183:
4.5 Additional GPFS management task
Page 184 and 185:
Copy the resulting kernel modules f
Page 186 and 187:
Then proceed to removing the GPFS p
Page 188 and 189:
5.1 Performance tuning cycle Tuning
Page 190 and 191:
ehave differently, number of worklo
Page 192 and 193:
Attention: If the data availability
Page 194 and 195:
The two impacting tuning areas for
Page 196 and 197:
Table 5-2 Overview of useful networ
Page 198 and 199:
# mmcrfs /gpfs /dev/gpfs -F -A no
Page 200 and 201:
## The value of the 'prefetchThread
Page 202 and 203:
5.2.5 Tuning the clients In a DM en
Page 204 and 205:
Registry parameter Default Suggeste
Page 206 and 207:
command nfsstat X X X nmon X X X X
Page 208 and 209:
The default Dconf configuration fil
Page 210 and 211:
FILES = ( /etc/gpfs/shmux /root/.ss
Page 212 and 213:
Detaching from and re-attaching to
Page 214 and 215:
you plan to run commands on subsets
Page 216 and 217:
In addition, dstat also has GPFS re
Page 218 and 219:
- Number of open()/create() gpfs ca
Page 220 and 221:
gpfsperf carefully, as gpfsperf is
Page 222 and 223:
Data rate was 204784.35 Kbytes/sec,
Page 224 and 225:
4. Using strace on Linux to analyze
Page 226 and 227:
Filename Description /var/mmfs/gen/
Page 228 and 229:
A.1 Automatic network installation
Page 230 and 231:
Tip 1: An entry in the arp cache is
Page 232 and 233:
Figure A-1 YaST Installation Server
Page 234 and 235:
Instead of using the BladeCenter®
Page 236 and 237:
AIX and Linux [p630n01][/etc/ssh]>
Page 238 and 239:
Example: A-9 Sample ~/.ssh/authoriz
Page 240 and 241:
A.8 SLES9 Installation on an IBM eS
Page 242 and 243:
NVSRAM version: None Transferred on
Page 244 and 245:
GPFS_data4 (271,463 GB) Associated
Page 246 and 247:
DETAILS Logical Drive name: GPFS_da
Page 248 and 249:
2. The metadata describing the esse
Page 250 and 251:
extent such a system can scale the
Page 252 and 253:
Archiving/Annotation Functionality
Page 254 and 255:
the physical disk. In case of cache
Page 256 and 257:
This configuration file will expect
Page 258 and 259:
Page 260 and 261:
Page 262 and 263:
How to get IBM Redbooks Help from I
Page 264 and 265:
DMC 13 dm-multipath 149 DNS 122 dra
Page 266 and 267:
nfsver 162 NLE 8, 10 no_root_squash
Page 268 and 269:
Page 272:
Configuration and Tuning GPFS for D
show all

Configuration and Tuning GPFS for Digital Media ... - IBM Redbooks

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?