Expert Oracle Exadata - Parent Directory

CHAPTER 9 RECOVERING EXADATASystem Volume FailureRecall that the first two disks in the storage cell contain the Linux operating system and are commonlyreferred to as the “system volumes.” Exadata protects these volumes using software mirroring throughthe O/S. Even so, certain situations may require you to recover these disks from backup. Some reasonsfor performing cell recovery would include:• System volumes (disks 1 and 2) fail simultaneously.• The boot partition is damaged beyond repair.• File systems become corrupted.• A patch installation or upgrade fails.If you find yourself in any of these situations, it may be necessary, or at least more expedient, torecover the system volumes from backup. As discussed earlier, Exadata automatically maintains abackup of the last good boot configuration using a 4GB internal USB flash drive called the CELLBOOTUSB flash drive. Recovering the system volumes using this internal USB flash disk is commonly referredto as the storage cell rescue procedure. The steps for performing the cell rescue procedure basicallyinvolve booting from the internal USB drive and following the prompts for the type of rescue you want toperform. By the way, since Exadata comes equipped with an ILOM (Integrated Lights Out Managementmodule), you can perform all cell recovery operations remotely, across the network. There is no need tostand in front of the rack to perform a full cell recovery from the internal USB flash disk.As you might imagine, this type of recovery should not be done without the assistance of OracleSupport. This section is not intended to be a step-by-step guide to cell recovery, so we’re not going to gointo all the details of cell recovery from the CELLBOOT USB flash disk here. The Oracle documentationshould be used for that, but we will take a look at what to consider before starting such a recovery.Cell Disks and Grid Disks: The rescue procedure restores the Linux systemvolumes only. Cell disks and their contents are not restored by the rescueprocedure. If these partitions are damaged they must be dropped and recreated.Once the grid disks are online, they can be added back to the ASM diskgroup and a subsequent rebalance will restore the data.ASM Redundancy: Recovering a storage cell from USB backup can potentiallycause the loss of all data on the system volumes. This includes your databasedata in the grid disks on these disk drives. If your ASM disk groups use Normalredundancy, we strongly recommend making a database backup beforeperforming cell recovery from USB disk. With ASM High redundancy, you havetwo mirror copies of all your data, so it is safe to perform cell recovery withouttaking database backups. Even so, we’d still take a backup if at all possible. Therecovery process does not destroy data volumes (cell/grid disks) unless youexplicitly choose to do so when prompted by the rescue procedure.Software and Patches: As of version 11.2 of the storage cell software, the rescueprocedure will restore the cell to its former state, patches included, when thebackup was taken. Also included in the restore are the network settings andSSH keys for the root, celladmin, and cellmonitor accounts. The alert settingsand SMTP configuration (including email addresses) for alert notification willnot be restored. Reconfiguring these settings is a trivial task and can be doneusing another cell to determine the correct settings to apply.300