AIX 5L Problem Determination - IBM Redbooks

More documents

Recommendations

Info

Display previously captured data The -o and -f options (write and read to or from user given data files) allow you to visualize the behavior of your machine in two independent steps. This consumes less resources during the problem-reproduction period. You can move the binary to another machine, because the binary file contains all data the sar command needs. # sar -o /tmp/sar.out 2 5 > /dev/null The previous command runs the sar command in the background, collects system activity data at two-second intervals for five intervals, and stores the (unformatted) sar data in the /tmp/sar.out file. The redirection of standard output is used to avoid a screen output. The following command extracts CPU information from the file and outputs a formatted report to standard output: # sar -f/tmp/sar.out AIX texmex 3 4 000691854C00 01/27/00 18:10:18 %usr %sys %wio %idle 18:10:20 9 2 0 88 18:10:22 13 10 0 76 18:10:24 37 4 0 59 18:10:26 8 2 0 90 18:10:28 20 3 0 77 Average 18 4 0 78 The captured binary data file keeps all information needed for the reports. Every possible sar report could, therefore, be investigated. System activity accounting and the cron daemon The sar command calls a process named sadc to access system data. Two shell scripts (/usr/lib/sa/sa1 and /usr/lib/sa/sa2) are structured to be run by the cron daemon and provide daily statistics and reports. Sample stanzas are included (but commented out) in the /var/spool/cron/crontabs/adm crontab file to specify when the cron daemon should run the shell scripts. The following lines show a modified crontab for the adm user. Only the comment characters for the data collections were removed: #========================================================= # SYSTEM ACTIVITY REPORTS # 8am-5pm activity reports every 20 mins during weekdays. # activity reports every an hour on Saturday and Sunday. # 6pm-7am activity reports every an hour during weekdays. # Daily summary prepared at 18:05. #========================================================= 0 8-17 * * 1-5 /usr/lib/sa/sa1 1200 3 & 236 IBM ^ Certification Study Guide - AIX 5L Problem Determination Tools and Techniques
0 * * * 0,6 /usr/lib/sa/sa1 & 0 18-7 * * 1-5 /usr/lib/sa/sa1 & 5 18 * * 1-5 /usr/lib/sa/sa2 -s 8:00 -e 18:01 -i 3600 -ubcwyaqvm & #========================================================= Collection of data in this manner is useful to characterize system usage over a period of time and to determine peak usage hours. Another useful feature of the sar command is that the output can be specific for the usage of each processor in a multiprocessor environment, as seen in the following output. The last line is an average output: # sar -P ALL 2 1 AIX client1 3 4 000BC6DD4C00 07/06/00 14:46:52 cpu %usr %sys %wio %idle 14:46:54 0 0 0 0 100 1 0 1 0 99 2 0 0 0 100 3 0 0 0 100 - 0 0 0 100 In general, if %usr plus %sys is consistently over 80 percent, then the system is CPU bound. System calls The sar -c command reports system calls. # sar -c 1 10 AIX server2 3 4 000FA17D4C00 06/30/00 09:33:04 scall/s sread/s swrit/s fork/s exec/s rchar/s wchar/s 09:33:05 1050 279 118 0.00 0.00 911220 5376749 09:33:06 186 19 74 0.00 0.00 3272 3226417 09:33:07 221 19 79 0.00 0.00 3272 3277806 09:33:08 2996 132 400 0.00 0.00 314800 2284933 09:33:09 3304 237 294 0.00 0.00 167733 848174 09:33:10 4186 282 391 0.00 0.00 228196 509414 09:33:11 1938 109 182 1.00 1.00 153703 1297872 09:33:12 3263 179 303 0.00 0.00 242048 1003364 09:33:13 2751 172 258 0.00 0.00 155082 693801 09:33:14 2827 187 285 0.00 0.00 174059 1155239 Chapter 10. Performance problem determination 237
Page 1:
ibm.com/redbooks IBM Certification
Page 4 and 5:
Note: Before using this information
Page 6 and 7:
3.5.1 The /etc/inittab file . . . .
Page 8 and 9:
7.2.1 Data relocation . . . . . . .
Page 10 and 11:
9.9.1 The lslicense command . . . .
Page 12 and 13:
x IBM ^ Certification Study Guide -
Page 14 and 15:
10-12 Performance tuning flowchart
Page 16 and 17:
8-7 Commonly used flags of the rout
Page 18 and 19:
Trademarks The following terms are
Page 20 and 21:
This publication does not replace p
Page 22 and 23:
Comments welcome Your comments are
Page 24 and 25:
1.1 Certification requirements To a
Page 26 and 27:
e. Verify that file systems are mou
Page 28 and 29:
Preparation for this exam is the to
Page 30 and 31:
2.1 Defining the problem The first
Page 32 and 33:
If the problem occurs on more than
Page 34 and 35:
2.4.1 Answers The following are the
Page 36 and 37:
3.1 A general overview of the boot
Page 38 and 39:
mainly to components on the motherb
Page 40 and 41:
Figure 3-3 Task selection menu in d
Page 42 and 43:
3.2.2 PCI systems next step is to s
Page 44 and 45:
3.3 Boot phase 1 Newer PCI architec
Page 46 and 47:
3.4 Boot phase 2 In boot phase 2, t
Page 48 and 49:
As mentioned, there are a lot of di
Page 50 and 51:
3.4.3 LED 518 product documentation
Page 52 and 53:
Figure 3-9 Boot phase 3 /etc/initta
Page 54 and 55:
3.5.2 LED 553 3.5.3 LED c31 3.5.4 L
Page 56 and 57:
3.5.5 pSeries servers A specific LE
Page 58 and 59:
When the system is gracefully shut
Page 60 and 61:
LED Reason/action c31 Define the co
Page 62 and 63:
3. A file system is being mounted b
Page 64 and 65:
3.9.1 Answers 3.10 Exercises The fo
Page 66 and 67:
4.1 Hardware basics RS/6000 and pSe
Page 68 and 69:
The most important fields in the pr
Page 70 and 71:
Figure 4-1 Main diagnostics menu Th
Page 72 and 73:
► Full testing requires that the
Page 74 and 75:
3. Place the CD-ROM into the drive.
Page 76 and 77:
Note: This service aid is only pres
Page 78 and 79:
2. Select Change Show use of an SSA
Page 80 and 81:
LED 299 A LED code of 299 shows tha
Page 82 and 83:
Figure 4-2 Format of the 103 code m
Page 84 and 85:
4.5 Quiz The following assessment q
Page 86 and 87:
4.5.1 Answers 4.6 Exercises The fol
Page 88 and 89:
5.1 Configuring the dump device Pri
Page 90 and 91:
5.2 Starting a system dump A user-i
Page 92 and 93:
A command status screen will be dis
Page 94 and 95:
If the system does not have a key s
Page 96 and 97:
The shconf command is invoked when
Page 98 and 99:
5.4 Increasing the size of the dump
Page 100 and 101:
Setting output device to /dev/rmt0.
Page 102 and 103:
Look at the time of the dump and th
Page 104 and 105:
Failure Causes SOFTWARE PROGRAM Rec
Page 106 and 107:
Using crash, you can examine: ► A
Page 108 and 109:
► ? or help[] Lists all subcomman
Page 110 and 111:
LR: .waitproc+d4 (0000edc4) 2ff3b38
Page 112 and 113:
DSIRR Data Storage Interrupt Reason
Page 114 and 115:
*prevthread:0xe6000000 *nextthread:
Page 116 and 117:
In this example, the VMM return cod
Page 118 and 119:
The snap command syntax is as follo
Page 120 and 121:
Flag Description -t Format Lists ea
Page 122 and 123:
Flag Description -p Device Temporar
Page 124 and 125:
Figure 5-3 Error log case study 2.
Page 126 and 127:
Figure 5-4 Case study Which of the
Page 128 and 129:
5.10.1 Answers 5.11 Exercises The f
Page 130 and 131:
6.1 The error daemon The error logg
Page 132 and 133:
To change the current log file, the
Page 134 and 135:
To process a report from the error
Page 136 and 137:
Flag Description -g Displays the AS
Page 138 and 139:
Flag Description -F FlagList Select
Page 140 and 141:
Probable Causes ERRDEMON STARTED AU
Page 142 and 143:
To display a detailed report of all
Page 144 and 145:
USER ID 0 0=SOFT IPL 1=HALT 2=TIME
Page 146 and 147:
Resource Class: planar Resource Typ
Page 148 and 149:
Flag Description -k ErrorID[,ErrorI
Page 150 and 151:
6.5.1 Setting up an accounting syst
Page 152 and 153:
6.6 The syslogd daemon The syslogd
Page 154 and 155:
Use the following message priority
Page 156 and 157:
3. A system operator accidentally d
Page 158 and 159:
Page 160 and 161:
7.1 LVM data The Logical Volume Man
Page 162 and 163:
7.2.1 Data relocation 7.2.2 Backup
Page 164 and 165:
AIX version level Verify that the v
Page 166 and 167:
This example shows that the volume
Page 168 and 169:
This error information displays the
Page 170 and 171:
hdisk0 Available 30-58-00-8,0 16 Bi
Page 172 and 173:
7.4 The AIX JFS 7.4.1 Creating a JF
Page 174 and 175:
7.4.2 Increasing the file system si
Page 176 and 177:
In many cases, restoration of the b
Page 178 and 179:
lv02 jfs 2 2 1 open/syncd /u/testfs
Page 180 and 181:
► Understanding enhanced journale
Page 182 and 183:
512 bytes) to minimize internal fra
Page 184 and 185:
Function JFS2 JFS Compression No Ye
Page 186 and 187:
7.4.10 Disk quota Pseudo file name
Page 188 and 189:
home: blocks in use: 0, limits (sof
Page 190 and 191:
7.5.3 Reducing and removing paging
Page 192 and 193:
Table 7-3 Commonly used flag of the
Page 194 and 195:
7.6.4 The rmlvcopy command The rmlv
Page 196 and 197:
Which of the following commands sho
Page 198 and 199:
Page 200 and 201:
8.1 Network interface problems If h
Page 202 and 203:
As you can see, the output shows th
Page 204 and 205:
3. If there is a default route, att
Page 206 and 207:
If you are using the route command
Page 208 and 209: address of the named server and try
Page 210 and 211: TOK: access control field = 0, fram
Page 212 and 213: 4. Initiate an iptrace (client, ser
Page 214 and 215: the nfs_max_threads parameter of th
Page 216 and 217: Flags Description -s host Records p
Page 218 and 219: 8.6 Quiz Table 8-8 Commonly used fl
Page 220 and 221: 8. Which of the following options w
Page 222 and 223: 8.6.1 Answers 8.7 Exercises The fol
Page 224 and 225: 9.1 User license problems If it is
Page 226 and 227: telnet: connect: A remote host did
Page 228 and 229: 9.2.4 Telnet error If when attempti
Page 230 and 231: user is limited and defined as an A
Page 232 and 233: 9.4.3 A full file system uid=0(root
Page 234 and 235: 9.5.1 Trace hook IDs The events tra
Page 236 and 237: your trace, the system may be traci
Page 238 and 239: 113 0.004261661 0.001518 unlock: lo
Page 240 and 241: If the sendmail command encounters
Page 242 and 243: 9.8 Printing Task SMIT fast path Co
Page 244 and 245: Action or function System V PowerPC
Page 246 and 247: Switching between subsystems The op
Page 248 and 249: Flag Description -s Subsystem Speci
Page 250 and 251: 9.10 Quiz Flag Description -O Optio
Page 252 and 253: 6. A user has the ability to ping t
Page 254 and 255: 232 IBM ^ Certification Study Guide
Page 256 and 257: Figure 10-1 General performance tun
Page 260 and 261: 10.1.2 The vmstat command The vmsta
Page 262 and 263: later). The method used in AIX Vers
Page 264 and 265: second. Large values indicate a CPU
Page 266 and 267: kernel routine records the process
Page 268 and 269: syncd 1 2 2 0 0 0 sh 1 1 1 0 0 0 sl
Page 270 and 271: The memory columns The memory colum
Page 272 and 273: The sr column The sr column shows t
Page 274 and 275: If you want to sort to the sixth co
Page 276 and 277: ► Workload Management Class (-W)
Page 278 and 279: schedtune limitations The schedtune
Page 280 and 281: schedtune parameters are reset by t
Page 282 and 283: Normal files are automatically mapp
Page 284 and 285: tty: tin tout avg-cpu: % user % sys
Page 286 and 287: When looking for performance proble
Page 288 and 289: those that do not. If you are runni
Page 290 and 291: The netstat -m command The netstat
Page 292 and 293: spec buf 1 0 0 0 128 128 0 locking
Page 294 and 295: Late Collision Errors The Late Coll
Page 296 and 297: The retrans column displays the num
Page 298 and 299: Figure 10-3 Web-based System Manage
Page 300 and 301: Superclasses A superclass is a clas
Page 302 and 303: ► Shared subclass: This subclass
Page 304 and 305: The sequence of attributes within a
Page 306 and 307: This manual assignment can be done
Page 308 and 309:
This defines three levels of privil
Page 310 and 311:
The class assignment is done by WLM
Page 312 and 313:
Figure 10-8 shows the SMIT panel wh
Page 314 and 315:
Figure 10-10 SMIT panel for rset re
Page 316 and 317:
10.6.1 The dbx command The dbx comm
Page 318 and 319:
Subcommand Description return Conti
Page 320 and 321:
15) phxent_dd [5 entries] 16) kbddd
Page 322 and 323:
10.8 Command summary The following
Page 324 and 325:
10.8.4 The nfsstat command 10.9 Qui
Page 326 and 327:
6. Which command will show system c
Page 328 and 329:
306 IBM ^ Certification Study Guide
Page 330 and 331:
11.1 Overview 11.1.1 Terminology 11
Page 332 and 333:
Note: Before installing a new set o
Page 334 and 335:
► APAR number, such as IY00301. I
Page 336 and 337:
► To install all filesets associa
Page 338 and 339:
Table 11-2 Commonly used flags of t
Page 340 and 341:
11.5 Quiz Flag Description -f Check
Page 342 and 343:
Page 344 and 345:
- A Web server The browser must be
Page 346 and 347:
Figure 12-2 HTTP filesets If you ar
Page 348 and 349:
- Remote computer Type the remote d
Page 350 and 351:
Figure 12-4 Documentation Library S
Page 352 and 353:
Page 354 and 355:
CISPR International Special Committ
Page 356 and 357:
I/O Input/Output I 2 C Inter Integr
Page 358 and 359:
OEM Original Equipment Manufacturer
Page 360 and 361:
SPM System Performance Measurement
Page 362 and 363:
Page 364 and 365:
Other resources These publications
Page 366 and 367:
Page 368 and 369:
phase1 figure 23 phase2 15, 24, 36
Page 370 and 371:
pc.statd 189 daemons shdaemon 73 te
Page 372 and 373:
host name resolution telnet login p
Page 374 and 375:
N name resolution diagnostic 185 pr
Page 376 and 377:
ecent CPU usage 241 recovering an i
Page 378 and 379:
System Resource Controller (SRC) 13
Page 380 and 381:
Page 384:
IBM Certification Study Guide - AIX
show all

AIX 5L Problem Determination - IBM Redbooks

Create successful ePaper yourself

Delete template?

Save as template?