Configuration and Tuning GPFS for Digital Media ... - IBM Redbooks

More documents

Recommendations

Info

Storage-Layer Work-Layer Communication-Layer Agents Video Audio mpeg1_ mepg4_trans. virage_vl vdx _convert flipfactory _transcode real _encode wave _rate_conv. wave _cutting mpeg1 _decoding mpeg1 _encoding mp3 wave _headroom wave _aiff_conv. bwf _injection cut _mpeg_data cut _pcm_data Legend Data controlled process Job controlled process Time or self controlled process Control & communication-process Storage system cm_control agent TQM delete agent Figure 1-11 ADMIRA - architecture and workflow Transfer from & to other systems such as FTP server or directories / disk pools ) make_zip agent Self managed cache (Content Manager) vc_control agent Bandwidth controlled cache (VideoCharger) Data Job ftp (put/get) agent Optional component WatchFolder put_to_archive agent copy_from _cache agent Disk cache with space management copy_to _cache agent get_from_archive if file not in cache Long-term archive (TSM) Based on this client-server design, ADMIRA can also be used in distributed environments as the communication of all components is processed over the LAN, MAN or WAN (see Figure 1-12 on page 25). Based on this client-server design, ADMIRA is a distributed system. Agents can run on different machines and platforms which are connected via LAN, MAN and WAN. New agents and new servers can be added during operation without stopping the system. This significantly increases the scalability and reliability. release agent access agent In addition, several ADMIRA systems can be linked together (i.e., for broadcasters with multiple, geographically distributed sites). A transfer agent synchronizes the ADMIRA instances and the consistency agent verifies the consistency of the different instances, i.e., after a loss of network connection. 24 Configuration and Tuning GPFS for Digital Media Environments Control of ADMIRA via jobs from external systems External Client Interface reply agent ADMIRA admin- / control-server, job- / system- & user management Control database (DB2) Comparison of ADMIRA with longterm archive and external systems consistency check transfer agent Data & job transfer to external Servers admira_system agent end_of_job agent Extenal ADMIRA sites External ADMIRA sites
Metadata Database Server Shared Storage (disk-cache) Applications / Clients Intranet Agents-Server on AIX, Linux or Windows Figure 1-12 ADMIRA in distributed environments For further information, send an email to: mailto:ADMIRA@de.ibm.com ADMIRA Job Control Server TSM + Tape-Library (Tape-Archive) Streaming- Server Shared Storage (disk-cache) 1.7.1 Open Enterprise System Virtualization (OESV) Applications / Clients Intranet Agents-Server on AIX, Linux or Windows WAN ADMIRA Job Control Server TSM + Tape-Library (Tape-Archive) IBM OESV is an OpenSource based, integrated solution which combines common infrastructure services as a file service (Samba) and print service (CUPS). Figure 1-13 on page 26 gives an introduction to OESV. Streaming- Server OESV combines external storage and server virtualization. External storage in combination with Linux-based grid functionality is mostly fault tolerant without using extra clustering software. Windows®, UNIX® and Linux file servers are bundled together into a redundant server-farm, which is exploited and reachable over virtual IP addresses, therefore providing the following advantages: ► Higher productivity and availability ► Less administration overhead Chapter 1. Introduction to digital media 25
Page 1: Configuration and Tuning GPFS for D
Page 4 and 5: Note: Before using this information
Page 6 and 7: 2.2.8 Quorum rules . . . . . . . .
Page 8 and 9: Abbreviations and acronyms . . . .
Page 10 and 11: Trademarks The following terms are
Page 12 and 13: Dag Wieers is a Linux and Open Sour
Page 14 and 15: Comments welcome Your comments are
Page 16 and 17: xiv Configuration and Tuning GPFS f
Page 18 and 19: 1.1 What is digital media? Digital
Page 20 and 21: Create Manage Distribute Transact F
Page 22 and 23: investments. And integration of dig
Page 24 and 25: 1.3.3 Broadcasting Table 1-2 Video
Page 26 and 27: 1.4 Broadcasting Within IBM this is
Page 28 and 29: 2. By using standard IT based compo
Page 30 and 31: Note: We are using the term offerin
Page 32 and 33: ► Streaming - Found in Digital Me
Page 34 and 35: These issues lead to the following
Page 36 and 37: ► Library Manager (LM) The LM pro
Page 38 and 39: ADMIRA does not provide the complet
Page 42 and 43: ► Maintenance without downtime IB
Page 44 and 45: LDAP The OpenLDAP service installed
Page 46 and 47: 30 Configuration and Tuning GPFS fo
Page 48 and 49: 2.1 GPFS history and evolution The
Page 50 and 51: 2.1.1 Why choose GPFS? This section
Page 52 and 53: RPC services Other Nodes Configurat
Page 54 and 55: 2.2.3 GPFS block allocation GPFS us
Page 56 and 57: a file (read or write) if it holds
Page 58 and 59: Token Token received received for f
Page 60 and 61: ► minMissedPingTimeout Minimum am
Page 62 and 63: lease missed lease renewal CfgMgr d
Page 64 and 65: Note: Whenever possible, we recomme
Page 66 and 67: 4 Request for accessing a file 1 no
Page 68 and 69: Data LUN0 In order to assure file a
Page 70 and 71: Data LUN0 2.2.8 Quorum rules 2 3’
Page 72 and 73: File system descriptor For maintain
Page 74 and 75: General Parallel File System Failur
Page 76 and 77: allocated at daemon start-up. The p
Page 78 and 79: Figure 2-19 GPFS prefetch 2.2.10 GP
Page 80 and 81: choices. Option two is using NSD on
Page 82 and 83: Application Node With GPFS Daemon A
Page 84 and 85: High SPEED IP INTERCONNECT GbE, 10G
Page 86 and 87: ► Application (compatibility, blo
Page 88 and 89: - DS4500: At the time this book was
Page 90 and 91:
command node influencing / node rel
Page 92 and 93:
command node influencing / node rel
Page 94 and 95:
performance impacting commands mmrp
Page 96 and 97:
80 Configuration and Tuning GPFS fo
Page 98 and 99:
3.1 General considerations for inst
Page 100 and 101:
System component # Type Feature Hos
Page 102 and 103:
2109-F32 (switch1) 2109-F32 (switch
Page 104 and 105:
Figure 3-4 DS4500 - Save the storag
Page 106 and 107:
► Configuration for data LUNs: -
Page 108 and 109:
Note: For performance and availabil
Page 110 and 111:
GPFS_metadata2 1 B Host Group GPFS-
Page 112 and 113:
Figure 3-8 DS4500 Storage Manager G
Page 114 and 115:
3.2.4 Servers zoning configuration
Page 116 and 117:
Figure 3-10 LPAR profile of p5 550
Page 118 and 119:
The second network will be used as
Page 120 and 121:
Once the Linux operating system has
Page 122 and 123:
Figure 3-14 SLES9-yast-setup - Inst
Page 124 and 125:
Figure 3-16 Final installation conf
Page 126 and 127:
For example, if storage (LUNs, mask
Page 128 and 129:
Important: If RDAC/MPP is used, a n
Page 130 and 131:
INITRD_MODULES="ipr lpfcdd" where
Page 132 and 133:
An easy way to verify that the crea
Page 134 and 135:
proc/mpp/DS4500_1/controllerA: tota
Page 136 and 137:
Type: Direct-Access ANSI SCSI revis
Page 138 and 139:
problem with the GPFS cluster if DN
Page 140 and 141:
3.3.5 Time synchronization Check th
Page 142 and 143:
Windows client time synchronization
Page 144 and 145:
► Save the changes to site.mcr fi
Page 146 and 147:
* Descriptions and downloads for th
Page 148 and 149:
The disk descriptor file contains a
Page 150 and 151:
The pagepool parameter changes the
Page 152 and 153:
Example 3-25 Output of mount comman
Page 154 and 155:
138 Configuration and Tuning GPFS f
Page 156 and 157:
4.1 Linux multipathing The term mul
Page 158 and 159:
The Figure 4-3 shows the RDAC drive
Page 160 and 161:
Note: We recommend that you save a
Page 162 and 163:
► Make sure the mppUpper module i
Page 164 and 165:
Attention: We have tested the devic
Page 166 and 167:
kernel 2.6 user space kernel space
Page 168 and 169:
The order of the device maps indica
Page 170 and 171:
devices { device { vendor "IBM " pr
Page 172 and 173:
use of classic remote shell (rsh),
Page 174 and 175:
4.4 NFS Windows network protocol th
Page 176 and 177:
If everything loaded correctly, you
Page 178 and 179:
Forcing the NFS version When mounti
Page 180 and 181:
First prevent the NFS server from s
Page 182 and 183:
4.5 Additional GPFS management task
Page 184 and 185:
Copy the resulting kernel modules f
Page 186 and 187:
Then proceed to removing the GPFS p
Page 188 and 189:
5.1 Performance tuning cycle Tuning
Page 190 and 191:
ehave differently, number of worklo
Page 192 and 193:
Attention: If the data availability
Page 194 and 195:
The two impacting tuning areas for
Page 196 and 197:
Table 5-2 Overview of useful networ
Page 198 and 199:
# mmcrfs /gpfs /dev/gpfs -F -A no
Page 200 and 201:
## The value of the 'prefetchThread
Page 202 and 203:
5.2.5 Tuning the clients In a DM en
Page 204 and 205:
Registry parameter Default Suggeste
Page 206 and 207:
command nfsstat X X X nmon X X X X
Page 208 and 209:
The default Dconf configuration fil
Page 210 and 211:
FILES = ( /etc/gpfs/shmux /root/.ss
Page 212 and 213:
Detaching from and re-attaching to
Page 214 and 215:
you plan to run commands on subsets
Page 216 and 217:
In addition, dstat also has GPFS re
Page 218 and 219:
- Number of open()/create() gpfs ca
Page 220 and 221:
gpfsperf carefully, as gpfsperf is
Page 222 and 223:
Data rate was 204784.35 Kbytes/sec,
Page 224 and 225:
4. Using strace on Linux to analyze
Page 226 and 227:
Filename Description /var/mmfs/gen/
Page 228 and 229:
A.1 Automatic network installation
Page 230 and 231:
Tip 1: An entry in the arp cache is
Page 232 and 233:
Figure A-1 YaST Installation Server
Page 234 and 235:
Instead of using the BladeCenter®
Page 236 and 237:
AIX and Linux [p630n01][/etc/ssh]>
Page 238 and 239:
Example: A-9 Sample ~/.ssh/authoriz
Page 240 and 241:
A.8 SLES9 Installation on an IBM eS
Page 242 and 243:
NVSRAM version: None Transferred on
Page 244 and 245:
GPFS_data4 (271,463 GB) Associated
Page 246 and 247:
DETAILS Logical Drive name: GPFS_da
Page 248 and 249:
2. The metadata describing the esse
Page 250 and 251:
extent such a system can scale the
Page 252 and 253:
Archiving/Annotation Functionality
Page 254 and 255:
the physical disk. In case of cache
Page 256 and 257:
This configuration file will expect
Page 258 and 259:
Page 260 and 261:
Page 262 and 263:
How to get IBM Redbooks Help from I
Page 264 and 265:
DMC 13 dm-multipath 149 DNS 122 dra
Page 266 and 267:
nfsver 162 NLE 8, 10 no_root_squash
Page 268 and 269:
Page 272:
Configuration and Tuning GPFS for D
show all

Configuration and Tuning GPFS for Digital Media ... - IBM Redbooks

Create successful ePaper yourself

Delete template?

Save as template?