VSAN-Troubleshooting-Reference-Manual

Recommendations

Info

Diagnostics and Troubleshooting Reference Manual – Virtual SAN Observations when a flash device fails In this example, the flash cache device (in this case an SSD) was removed from the disk group. The events were similar to what was observed with a magnetic disk removal. Virtual SAN determines that the device has a permanent failure, enters APD state, and the device being marked as permanently inaccessible. Note however that all the magnetic disks in the disk group also go offline. When a flash cache device in a disk group is impacted by a failure, the whole of the disk group is impacted. The disk group status in the vSphere web client shows the overall disk group is now “Unhealthy”. The status of the magnetic disks in the same disk group shows “Flash disk down”. If the timeout expires before the original device has been reinserted in the host, the rebuilding of components in the disk group takes place. V M W A R E S T O R A G E B U D O C U M E N T A T I O N / 1 6 0
Diagnostics and Troubleshooting Reference Manual – Virtual SAN Observations when a storage controller fails In this particular case, an LSI MegaRAID storage controller had issues when it was using old driver/firmware in Virtual SAN. The following are vmkernel.log samples taken from each of the hosts in the Virtual SAN cluster. Controller resets: 2014-08-24T17:00:25.940Z cpu29:33542)megasas: Found FW in FAULT state, will reset adapter. 2014-08-24T17:00:25.940Z cpu29:33542)megaraid_sas: resetting fusion adapter. I/Os fail due to controller issue (SCSI write is Cmd 0x2a): 2014-08-24T17:00:25.940Z cpu34:9429858)NMP: nmp_ThrottleLogForDevice:2321: Cmd 0x2a (0x4136803d32c0, 0) to dev "naa.50015178f3636429" on path "vmhba0:C0:T4:L0" Failed: H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL 2014-08-24T17:00:25.940Z cpu34:9429858)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "naa.50015178f3636429" state in doubt; requested fast path state update... 2014-08-24T17:00:25.940Z cpu34:9429858)ScsiDeviceIO: 2324: Cmd(0x41368093bd80) 0x2a, CmdSN 0x648c4f3b from world 0 to dev "naa.50015178f3636429" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. 2014-08-24T17:00:25.940Z cpu34:9429858)ScsiDeviceIO: 2324: Cmd(0x4136e17d15c0) 0x2a, CmdSN 0x648c4ee8 from world 0 to dev "naa.50015178f3636429" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. 2014-08-24T17:00:25.940Z cpu34:9429858)NMP: nmp_ThrottleLogForDevice:2321: Cmd 0x2a (0x4136e2370d40, 0) to dev "naa.50015178f3636429" on path "vmhba0:C0:T4:L0" Failed: H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL 2014-08-24T17:00:25.940Z cpu34:9429858)ScsiDeviceIO: 2324: Cmd(0x41370c3043c0) 0x2a, CmdSN 0x648c4f3a from world 0 to dev "naa.50015178f3636429" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. 2014-08-24T17:00:25.940Z cpu34:9429858)ScsiDeviceIO: 2324: Cmd(0x4136e17d4680) 0x2a, CmdSN 0x648c4eeb from world 0 to dev "naa.50015178f3636429" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. 2014-08-24T17:00:25.940Z cpu34:9429858)NMP: nmp_ThrottleLogForDevice:2321: Cmd 0x2a (0x4136e07e1700, 0) to dev "naa.50015178f3636429" on path "vmhba0:C0:T4:L0" Failed: H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL 2014-08-24T17:00:25.940Z cpu34:9429858)NMP: nmp_ThrottleLogForDevice:2321: Cmd 0x28 (0x4136e884c500, 0) to dev "naa.5000c500583c4b1f" on path "vmhba0:C0:T6:L0" Failed: H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL 3) When the MegaRAID firmware is resetting, it simply takes too long for Virtual SAN to hold off, and eventually fails because it simply took to long for the controller to come back online. The retries are visible in the logs, as well as the maximum number of kernel-level retries exceeded: 2014-08-24T17:00:30.845Z cpu38:33542)megasas: Waiting for FW to come to ready state [ … ] 2014-08-24T17:00:30.912Z cpu20:33167)LSOMCommon: IORETRYCompleteIO:389: Throttled: 0x413701b8cc40 IO type 265 (READ) isOdered:NO since 30001 msec status Maximum kernellevel retries exceeded V M W A R E S T O R A G E B U D O C U M E N T A T I O N / 1 6 1
Page 1 and 2:
Diagnostics and Troubleshooting Gui
Page 3 and 4:
Diagnostics and Troubleshooting Ref
Page 5 and 6:
Page 7 and 8:
Page 9 and 10:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17 and 18:
Page 19 and 20:
Page 21 and 22:
Page 23 and 24:
Page 25 and 26:
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
Page 37 and 38:
Page 39 and 40:
Page 41 and 42:
Page 43 and 44:
Page 45 and 46:
Page 47 and 48:
Page 49 and 50:
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
Page 59 and 60:
Page 61 and 62:
Page 63 and 64:
Page 65 and 66:
Page 67 and 68:
Page 69 and 70:
Page 71 and 72:
Page 73 and 74:
Page 75 and 76:
Page 77 and 78:
Page 79 and 80:
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110: Diagnostics and Troubleshooting Ref
Page 159: Diagnostics and Troubleshooting Ref
Page 211 and 212:
Page 213 and 214:
Page 215 and 216:
Page 217 and 218:
Page 219 and 220:
Page 221 and 222:
Page 223 and 224:
Page 225 and 226:
Page 227 and 228:
Page 229 and 230:
Page 231 and 232:
Page 233 and 234:
Page 235 and 236:
Page 237 and 238:
Page 239 and 240:
Page 241 and 242:
Page 243 and 244:
Page 245 and 246:
Page 247 and 248:
Page 249 and 250:
Page 251 and 252:
Page 253 and 254:
Page 255 and 256:
Page 257 and 258:
Page 259 and 260:
Page 261 and 262:
Page 263 and 264:
Page 265 and 266:
Page 267 and 268:
Page 269 and 270:
Page 271 and 272:
Page 273 and 274:
Page 275 and 276:
Page 277 and 278:
Page 279 and 280:
Page 281 and 282:
Page 283 and 284:
Page 285 and 286:
Page 287 and 288:
Page 289 and 290:
Page 291 and 292:
Page 293 and 294:
Page 295:
show all

VSAN-Troubleshooting-Reference-Manual

Create successful ePaper yourself

Delete template?

Save as template?