Disks can be designated as global hot spares. If a drive in an array fails, ZFS will automatically use available configured global hot spares to replace the failed drive. This speeds rebuilding of the failed drive and lessens the likelihood that additional drives in the same array will fail before the failed drive has been rebuilt.
Here are some sample configurations:
12 3TB drive configured as 11 drives in raidz2 (RAID6) + 1 hot spare drive
11 drives in raidz2 configuration = 2.7TB * 11 – (2 * 2.7TB) = 24TB usable space in the storage pool
12 3TB drives configured as 3 raidz2 (RAID6) arrays
4 drives in raidz2 configuration = 2.7TB * 4 – (2 * 2.7TB) = 5.4TB * 3 arrays = 16TB usable space in the storage pool
12 3TB drives configured as 2 raidz (RAID5) arrays in a mirror
6 3TB drives in raidz (RAID5) = 2.7TB *6 – (1 * 2.7TB) = 13.5TB usable space in the storage pool
Mirror of 6 3TB drives in raidz (RAID5) = 2.7TB *6 – (1 * 2.7TB) = 0TB (mirror)
Example 1 uses the same number of drives as example 2. However, in example 1 the usable space is 24TB, and there is one global hot spare available in the chassis.
In example 2, only 16TB of usable space is available, and there is no room for a global hot spare. This is because example 1 makes more efficient use of the drive space by only using two disks worth of capacity to parity information for the entire array, while example 2 uses six drives worth of capacity to parity information due to the smaller size of each array.
Example 3 provides a a full mirror of the array, thereby offering more redundancy and protecting against silent data corruption. However, this comes at the expense of usable disk space. Further, this allows up to two disks to fail in any one of the arrays, while simultaneously allowing up to one disk to fail in the second array, before the storage pool data is lost.
In summary, example 1 achieves both a large amount of usable storage, as well as a good measure of reliability and performance. It also tolerates the failure of any two drives in the array, while still making a global hot spare available.