AIX 5L Problem Determination - IBM Redbooks

More documents

Recommendations

Info

kernel routine records the process ID and the address of the instruction executing when the interrupt occurred and this information is used by the tprof command. The tprof command also records whether the process counter is in the kernel address space, the user address space, or the shared library address space. In AIX 5L Version 5.1, the tprof command has been enhanced to do subroutine or method level profiling for Java applications. The Java Virtual Machine Profiling Interface (JVMPI), a new feature supported by Java 1.2 or later, has been enhanced to do class and method level profiling for Java applications. The -j flag was added to tprof to enable profiling for Java applications. The tprof summary CPU utilization report A summary ASCII report with the suffix .all is always produced. If no program is specified, the report is named __prof.all. If a program is specified, the report is named __program.all. This report contains an estimate of the amount of CPU time spent in each process that was executing while the tprof command was monitoring the system. This report also contains an estimate of the amount of CPU time spent in each of the three address spaces. The files containing the reports are left in the working directory. All files created by the tprof command are prefixed by two underscores (___). In the following example, a generic report generated an output of: # tprof -x sleep 30 Starting Trace now Starting sleep 30 Wed Jun 28 14:58:58 2000 System: AIX server3 Node: 4 Machine: 000BC6DD4C00 Trace is done now 30.907 secs in measured interval * Samples from __trc_rpt2 * Reached second section of __trc_rpt2 In this case, sleep 30 points out to the tprof command to run for 30 seconds. The total column The total column in __prof.all is of interest. The first section indicates the use of ticks on a per-process basis. Process PID TID Total Kernel User Shared Other ======= === === ===== ====== ==== ====== ===== wait 516 517 3237 3237 0 0 0 tctestprg 14746 13783 3207 1 3206 0 0 tctestprg 13730 17293 3195 0 3195 0 0 244 IBM ^ Certification Study Guide - AIX 5L Problem Determination Tools and Techniques
wait 1032 1033 3105 3105 0 0 wait 1290 1291 138 138 0 0 0 swapper 0 3 10 7 3 0 0 tprof 14156 5443 6 3 3 0 0 trace 16000 14269 3 3 0 0 0 syncd 3158 4735 2 2 0 0 0 tprof 5236 16061 2 2 0 0 0 gil 2064 2839 1 1 0 0 0 gil 2064 3097 1 1 0 0 trace 15536 14847 1 1 0 0 0 sh 14002 16905 1 1 0 0 0 sleep 14002 16905 1 1 0 0 0 ======= === === ===== ====== ==== ====== ===== Total 12910 6503 6407 0 0 Each tick is 1/100 second. You can calculate the total amount of available ticks; for example, 30 seconds, times 100 ticks, make a total of 3000 ticks. This is according to theory, but when looking at the output, there are over 12000 total ticks. This is because the test system is a 4-way F50, so the available ticks are calculated in the following way: Time (in seconds) x Number of available CPUs x 100 The user column If the user column shows high values, application tuning might be necessary. In the output, you can see that both tctestprgs used about 3200 ticks, which is around 25 percent of the total number of available ticks. This is confirmed with ps auxwww output: # ps auxwww USER PID %CPU %MEM SZ RSS TTY STAT STIME TIME COMMAND root 14020 25.0 0.0 300 320 pts/1 A 15:23:55 16:45 ./tctestprg root 12280 25.0 0.0 300 320 pts/1 A 15:23:57 16:43 ./tctestprg The freq column The second section of the report includes the freq column, which has the total amount of ticks used by a specified type of process. Here the ticks used by all three wait processes are added together, and the two tctestprgs are added together. The total workload produced by one type of process is shown below (as well as the number of instances of the processes that are running): Process FREQ Total Kernel User Shared Other ======= === ===== ====== ==== ====== ===== wait 3 6480 6480 0 0 0 tctestprg 2 6402 1 6401 0 0 swapper 1 10 7 3 0 0 tprof 2 8 5 3 0 0 trace 2 4 4 0 0 0 gil 2 2 2 0 0 0 Chapter 10. Performance problem determination 245
Page 1:
ibm.com/redbooks IBM Certification
Page 4 and 5:
Note: Before using this information
Page 6 and 7:
3.5.1 The /etc/inittab file . . . .
Page 8 and 9:
7.2.1 Data relocation . . . . . . .
Page 10 and 11:
9.9.1 The lslicense command . . . .
Page 12 and 13:
x IBM ^ Certification Study Guide -
Page 14 and 15:
10-12 Performance tuning flowchart
Page 16 and 17:
8-7 Commonly used flags of the rout
Page 18 and 19:
Trademarks The following terms are
Page 20 and 21:
This publication does not replace p
Page 22 and 23:
Comments welcome Your comments are
Page 24 and 25:
1.1 Certification requirements To a
Page 26 and 27:
e. Verify that file systems are mou
Page 28 and 29:
Preparation for this exam is the to
Page 30 and 31:
2.1 Defining the problem The first
Page 32 and 33:
If the problem occurs on more than
Page 34 and 35:
2.4.1 Answers The following are the
Page 36 and 37:
3.1 A general overview of the boot
Page 38 and 39:
mainly to components on the motherb
Page 40 and 41:
Figure 3-3 Task selection menu in d
Page 42 and 43:
3.2.2 PCI systems next step is to s
Page 44 and 45:
3.3 Boot phase 1 Newer PCI architec
Page 46 and 47:
3.4 Boot phase 2 In boot phase 2, t
Page 48 and 49:
As mentioned, there are a lot of di
Page 50 and 51:
3.4.3 LED 518 product documentation
Page 52 and 53:
Figure 3-9 Boot phase 3 /etc/initta
Page 54 and 55:
3.5.2 LED 553 3.5.3 LED c31 3.5.4 L
Page 56 and 57:
3.5.5 pSeries servers A specific LE
Page 58 and 59:
When the system is gracefully shut
Page 60 and 61:
LED Reason/action c31 Define the co
Page 62 and 63:
3. A file system is being mounted b
Page 64 and 65:
3.9.1 Answers 3.10 Exercises The fo
Page 66 and 67:
4.1 Hardware basics RS/6000 and pSe
Page 68 and 69:
The most important fields in the pr
Page 70 and 71:
Figure 4-1 Main diagnostics menu Th
Page 72 and 73:
► Full testing requires that the
Page 74 and 75:
3. Place the CD-ROM into the drive.
Page 76 and 77:
Note: This service aid is only pres
Page 78 and 79:
2. Select Change Show use of an SSA
Page 80 and 81:
LED 299 A LED code of 299 shows tha
Page 82 and 83:
Figure 4-2 Format of the 103 code m
Page 84 and 85:
4.5 Quiz The following assessment q
Page 86 and 87:
4.5.1 Answers 4.6 Exercises The fol
Page 88 and 89:
5.1 Configuring the dump device Pri
Page 90 and 91:
5.2 Starting a system dump A user-i
Page 92 and 93:
A command status screen will be dis
Page 94 and 95:
If the system does not have a key s
Page 96 and 97:
The shconf command is invoked when
Page 98 and 99:
5.4 Increasing the size of the dump
Page 100 and 101:
Setting output device to /dev/rmt0.
Page 102 and 103:
Look at the time of the dump and th
Page 104 and 105:
Failure Causes SOFTWARE PROGRAM Rec
Page 106 and 107:
Using crash, you can examine: ► A
Page 108 and 109:
► ? or help[] Lists all subcomman
Page 110 and 111:
LR: .waitproc+d4 (0000edc4) 2ff3b38
Page 112 and 113:
DSIRR Data Storage Interrupt Reason
Page 114 and 115:
*prevthread:0xe6000000 *nextthread:
Page 116 and 117:
In this example, the VMM return cod
Page 118 and 119:
The snap command syntax is as follo
Page 120 and 121:
Flag Description -t Format Lists ea
Page 122 and 123:
Flag Description -p Device Temporar
Page 124 and 125:
Figure 5-3 Error log case study 2.
Page 126 and 127:
Figure 5-4 Case study Which of the
Page 128 and 129:
5.10.1 Answers 5.11 Exercises The f
Page 130 and 131:
6.1 The error daemon The error logg
Page 132 and 133:
To change the current log file, the
Page 134 and 135:
To process a report from the error
Page 136 and 137:
Flag Description -g Displays the AS
Page 138 and 139:
Flag Description -F FlagList Select
Page 140 and 141:
Probable Causes ERRDEMON STARTED AU
Page 142 and 143:
To display a detailed report of all
Page 144 and 145:
USER ID 0 0=SOFT IPL 1=HALT 2=TIME
Page 146 and 147:
Resource Class: planar Resource Typ
Page 148 and 149:
Flag Description -k ErrorID[,ErrorI
Page 150 and 151:
6.5.1 Setting up an accounting syst
Page 152 and 153:
6.6 The syslogd daemon The syslogd
Page 154 and 155:
Use the following message priority
Page 156 and 157:
3. A system operator accidentally d
Page 158 and 159:
Page 160 and 161:
7.1 LVM data The Logical Volume Man
Page 162 and 163:
7.2.1 Data relocation 7.2.2 Backup
Page 164 and 165:
AIX version level Verify that the v
Page 166 and 167:
This example shows that the volume
Page 168 and 169:
This error information displays the
Page 170 and 171:
hdisk0 Available 30-58-00-8,0 16 Bi
Page 172 and 173:
7.4 The AIX JFS 7.4.1 Creating a JF
Page 174 and 175:
7.4.2 Increasing the file system si
Page 176 and 177:
In many cases, restoration of the b
Page 178 and 179:
lv02 jfs 2 2 1 open/syncd /u/testfs
Page 180 and 181:
► Understanding enhanced journale
Page 182 and 183:
512 bytes) to minimize internal fra
Page 184 and 185:
Function JFS2 JFS Compression No Ye
Page 186 and 187:
7.4.10 Disk quota Pseudo file name
Page 188 and 189:
home: blocks in use: 0, limits (sof
Page 190 and 191:
7.5.3 Reducing and removing paging
Page 192 and 193:
Table 7-3 Commonly used flag of the
Page 194 and 195:
7.6.4 The rmlvcopy command The rmlv
Page 196 and 197:
Which of the following commands sho
Page 198 and 199:
Page 200 and 201:
8.1 Network interface problems If h
Page 202 and 203:
As you can see, the output shows th
Page 204 and 205:
3. If there is a default route, att
Page 206 and 207:
If you are using the route command
Page 208 and 209:
address of the named server and try
Page 210 and 211:
TOK: access control field = 0, fram
Page 212 and 213:
4. Initiate an iptrace (client, ser
Page 214 and 215:
the nfs_max_threads parameter of th
Page 216 and 217: Flags Description -s host Records p
Page 218 and 219: 8.6 Quiz Table 8-8 Commonly used fl
Page 220 and 221: 8. Which of the following options w
Page 222 and 223: 8.6.1 Answers 8.7 Exercises The fol
Page 224 and 225: 9.1 User license problems If it is
Page 226 and 227: telnet: connect: A remote host did
Page 228 and 229: 9.2.4 Telnet error If when attempti
Page 230 and 231: user is limited and defined as an A
Page 232 and 233: 9.4.3 A full file system uid=0(root
Page 234 and 235: 9.5.1 Trace hook IDs The events tra
Page 236 and 237: your trace, the system may be traci
Page 238 and 239: 113 0.004261661 0.001518 unlock: lo
Page 240 and 241: If the sendmail command encounters
Page 242 and 243: 9.8 Printing Task SMIT fast path Co
Page 244 and 245: Action or function System V PowerPC
Page 246 and 247: Switching between subsystems The op
Page 248 and 249: Flag Description -s Subsystem Speci
Page 250 and 251: 9.10 Quiz Flag Description -O Optio
Page 252 and 253: 6. A user has the ability to ping t
Page 254 and 255: 232 IBM ^ Certification Study Guide
Page 256 and 257: Figure 10-1 General performance tun
Page 258 and 259: Display previously captured data Th
Page 260 and 261: 10.1.2 The vmstat command The vmsta
Page 262 and 263: later). The method used in AIX Vers
Page 264 and 265: second. Large values indicate a CPU
Page 268 and 269: syncd 1 2 2 0 0 0 sh 1 1 1 0 0 0 sl
Page 270 and 271: The memory columns The memory colum
Page 272 and 273: The sr column The sr column shows t
Page 274 and 275: If you want to sort to the sixth co
Page 276 and 277: ► Workload Management Class (-W)
Page 278 and 279: schedtune limitations The schedtune
Page 280 and 281: schedtune parameters are reset by t
Page 282 and 283: Normal files are automatically mapp
Page 284 and 285: tty: tin tout avg-cpu: % user % sys
Page 286 and 287: When looking for performance proble
Page 288 and 289: those that do not. If you are runni
Page 290 and 291: The netstat -m command The netstat
Page 292 and 293: spec buf 1 0 0 0 128 128 0 locking
Page 294 and 295: Late Collision Errors The Late Coll
Page 296 and 297: The retrans column displays the num
Page 298 and 299: Figure 10-3 Web-based System Manage
Page 300 and 301: Superclasses A superclass is a clas
Page 302 and 303: ► Shared subclass: This subclass
Page 304 and 305: The sequence of attributes within a
Page 306 and 307: This manual assignment can be done
Page 308 and 309: This defines three levels of privil
Page 310 and 311: The class assignment is done by WLM
Page 312 and 313: Figure 10-8 shows the SMIT panel wh
Page 314 and 315: Figure 10-10 SMIT panel for rset re
Page 316 and 317:
10.6.1 The dbx command The dbx comm
Page 318 and 319:
Subcommand Description return Conti
Page 320 and 321:
15) phxent_dd [5 entries] 16) kbddd
Page 322 and 323:
10.8 Command summary The following
Page 324 and 325:
10.8.4 The nfsstat command 10.9 Qui
Page 326 and 327:
6. Which command will show system c
Page 328 and 329:
306 IBM ^ Certification Study Guide
Page 330 and 331:
11.1 Overview 11.1.1 Terminology 11
Page 332 and 333:
Note: Before installing a new set o
Page 334 and 335:
► APAR number, such as IY00301. I
Page 336 and 337:
► To install all filesets associa
Page 338 and 339:
Table 11-2 Commonly used flags of t
Page 340 and 341:
11.5 Quiz Flag Description -f Check
Page 342 and 343:
Page 344 and 345:
- A Web server The browser must be
Page 346 and 347:
Figure 12-2 HTTP filesets If you ar
Page 348 and 349:
- Remote computer Type the remote d
Page 350 and 351:
Figure 12-4 Documentation Library S
Page 352 and 353:
Page 354 and 355:
CISPR International Special Committ
Page 356 and 357:
I/O Input/Output I 2 C Inter Integr
Page 358 and 359:
OEM Original Equipment Manufacturer
Page 360 and 361:
SPM System Performance Measurement
Page 362 and 363:
Page 364 and 365:
Other resources These publications
Page 366 and 367:
Page 368 and 369:
phase1 figure 23 phase2 15, 24, 36
Page 370 and 371:
pc.statd 189 daemons shdaemon 73 te
Page 372 and 373:
host name resolution telnet login p
Page 374 and 375:
N name resolution diagnostic 185 pr
Page 376 and 377:
ecent CPU usage 241 recovering an i
Page 378 and 379:
System Resource Controller (SRC) 13
Page 380 and 381:
Page 384:
IBM Certification Study Guide - AIX
show all

AIX 5L Problem Determination - IBM Redbooks

Create successful ePaper yourself

Delete template?

Save as template?