VSAN-Troubleshooting-Reference-Manual

Diagnostics and Troubleshooting Reference Manual – Virtual SAN
What is I/O per second (IOPS)?
IOPS gives a measure of number of Input/Output Operations Per Second of a storage
system. An I/O operation is typically a read or a write operation and a size. I/O size
can vary between a few bytes and several megabytes.
If high IOPS is observed in Virtual SAN, it does not necessarily mean that we have a
problem. It could simply mean that we are using the storage to its maximum. For
example, a disk clone operation may try to use all available IOPS to complete the
operation in least possible time. 
 Similarly, a low IOPS value does not mean that
there is an immediate problem either; it could simply be that the I/O sizes are very
large, and because of the large I/O sizes, it implies that we do fewer IOPS. Typically
large numbers for IOPS is better.
What is bandwidth?
Bandwidth (or thoroughput) measures the data rate or throughput that a storage
device is capable of, or to put it another way, how much data is being transferred.
IOPS and bandwidth are related. When small I/O sizes (e.g. 4KB) are involved, a disk
device may hit the maximum IOPS ceiling before exhausting the available bandwidth
provided by the device, or controller, or the underlying physical link. Conversely,
for large I/O sizes (e.g. 1MB) the bandwidth may become a limiting factor before the
maximum IOPS of a device or controller is reached. Larger numbers for bandwidth
are better.
When troubleshooting storage performance, look at IOPS, I/O sizes, outstanding I/O,
and latency to get a complete picture. Throughput and latency increase in an almost
linear fashion as the size of your I/O increases, while at the same time the total
number of I/O (IOPS) will decrease.
What is congestion?
Congestion in Virtual SAN happens when the lower layers fail to keep up with the
I/O rate of higher layers.
For example, if virtual machines are performing a lot of write operations, it could
lead to filling up of write buffers on the flash cache device. These buffers have to be
destaged to magnetic disks in hybrid configurations. However, this de-staging can
only be done at a rate at which the magnetic disks in a hybrid configuration can
handle, much slower typically than flash device performance. This can causes
Virtual SAN to artificially introduce latencies in the virtual machines in order to
slow down writes to the flash device so that write buffers can be freed up.
Other reasons for congestion could be related to faulty hardware, bad or
misbehaving drivers/firmware or insufficient I/O controller queue depths.
V M W A R E S T O R A G E B U D O C U M E N T A T I O N / 2 0 7