Expert Oracle Exadata - Parent Directory

CHAPTER 5 EXADATA SMART FLASH CACHE1.125G of its addressable space for its own purposes, yielding a usable space of 22.875G per module. Thisyields a total of 366G of usable flash storage per storage cell. That’s over a terabyte (TB) on the smallestquarter rack configuration and over 5 TB on a full rack.The F20 cards also include an energy storage module (ESM), sometimes also referred to as a SuperCapacitor. You can think of it as a battery, although it actually uses a capacitor, whereas a traditionalbattery relies on chemical reactions. At any rate, the purpose of the ESM is to provide enough power toflush any data in its volatile memory to the nonvolatile flash memory in the case of a sudden power loss.The volatile memory consists of 64M of DRAM per FMod and is used to buffer writes to the nonvolatileflash memory. You should note that if the ESM fails, writes will by-pass the DRAM and be sent directly tothe flash memory. As a result, write speeds will be significantly decreased in the case of ESM failure. Thisdoesn’t present a major issue when the cards are configured as ESFC, since the Oracle storage softwaretreats the ESFC as a write-through cache and therefore writes directly to disk, bypassing the cache. It canbe an issue though if the cards are used as Flash Disks. Sun recommends that the ESM modules bereplaced every two years as the stored energy degrades over time (like a battery). The ILOM systemmanagement firmware monitors ESM usage and issues warnings as the ESM approaches the end of itslifespan. See the Sun Flash Accelerator F20 PCIe Card User’s Guide for further details.According to the Sun documentation, the storage servers allow PCIe cards to be replaced while thesystem is running. However, the Oracle Exadata Storage Software User’s Guide recommends poweringdown the storage servers before replacing one of these cards. Fortunately, you can accomplish thiswithout experiencing an outage, as ASM redundancy allows entire storage cells to be offline withoutaffecting the databases they are supporting. Note that replacing one of the F20 cards should not requireany reconfiguration. Figure 5-1 shows a conceptual diagram of one of the Flash Accelerator cards.126