Adaptive RAID-5 Erasure Coding with the Express Storage Architecture in vSAN 8
The ability to deliver RAID-5/6 space efficiency with the performance of RAID-1 in the new vSAN Express Storage Architecture (ESA) changes how our vSAN customers improve their capacity usage without any compromise in performance. But the improvements with space-efficient erasure coding do not stop there. The ESA in vSAN 8 introduces new capabilities exclusive to the use of RAID-5 that is sure to help customers increase the effective storage capacity of their vSAN clusters while making it easier to manage.
RAID-5 Using the vSAN Original Storage Architecture
The Original Storage Architecture (OSA) in previous versions of vSAN and vSAN 8 provided RAID-5 erasure coding through a 3+1 scheme. Meaning that the data is written as a stripe consisting of 3 data bits and 1 parity bit, across at minimum, 4 hosts. This scheme, as shown in Figure 1, can tolerate a single host failure (FTT=1) while maintaining the availability of the data.
Figure 1. RAID-5 erasure coding when using the vSAN Original Storage Architecture
This scheme, which consumes 1.33x the capacity of the primary data, was a space-efficient way to store data resiliently, but it had performance trade-offs and required at least 4 hosts in a cluster to support it.
Two New RAID-5 Erasure Coding Schemes in the ESA
The ESA in vSAN 8 replaces the 3+1 scheme above with two new RAID-5 erasure codes.
Option #1: FTT=1 using RAID-5 in a 4+1 scheme
As shown in Figure 2, this writes the data in a VM as a stripe consisting of 4 data bits and 1 parity bit, across a minimum of 5 hosts. It consumes just 1.25x the capacity of the primary data making it extremely space efficient without any performance tradeoff but requires 5 hosts to store the data resiliently.
Figure 2. RAID-5 erasure coding when using the vSAN Express Storage Architecture in a larger cluster
Option #2: FTT=1 using RAID-5 in a 2+1 scheme
As shown in Figure 3, this writes the data in a VM as a stripe consisting of 2 data bits and 1 parity bit, across a minimum of 3 hosts. It consumes 1.5x the capacity of the primary data, delivering guaranteed space efficiency of resilient data across just 3 hosts.
Figure 3. RAID-5 erasure coding when using the vSAN Express Storage Architecture in a small cluster
The 2+1 scheme opens all new opportunities to reduce capacity usage for smaller vSAN clusters that relied on RAID-1 mirroring for data resilience - which required 2x the capacity of the primary data.
Imagine a 3-host cluster that had VMs using FTT=1 using RAID-1 consuming 200TB of storage to ensure data resilience. In this same configuration, the ESA in vSAN 8 could apply an FTT=1 using RAID-5, and consume just 150TB of storage to ensure data resilience, but at the performance of RAID-1. In this example, that is a guaranteed savings of 50TB with the change of a storage policy. Factor in the ability to use all storage devices for capacity in the ESA, as well as its improved compression rates, and the cost savings that the ESA provides could be significant.
Recommendation: To provide even higher levels of resilience, consider standardizing on FTT=2 using RAID-6 in your clusters with 7 or more hosts. The vSAN ESA will give you high levels of resilience, and high levels of space efficiency without compromising performance.
Adaptive RAID-5
Perhaps the most creative aspect of this feature is vSAN's ability to automatically adapt the RAID-5 erasure code to best suit the host count of the cluster. The vSAN ESA presents a single RAID-5 storage policy rule for you to select, and will adapt the RAID-5 scheme based on the host count of the cluster. Additionally, it will determine which RAID-5 scheme to use not by the minimum hosts required, but by the minimum hosts recommended to ensure there is a spare fault domain (host) whenever possible. Let's look at the animation in Figure 4 to understand how objects with a storage policy using RAID-5 would behave.
- Cluster with 6 hosts. The objects will use the 4+1 scheme, spreading the stripe with parity across 5 hosts, while still having one spare fault domain to regain the prescribed level of resilience under subsequent host maintenance mode or failure conditions.
- Cluster reduced to 5 hosts. The object will continue to maintain this fully resilient 4+1 scheme across 5 hosts for 24 hours. After this period, it will reconfigure that object to use the 2+1 scheme, so that it has at least one spare fault domain to regain the prescribed level of resilience under subsequent maintenance mode or failure conditions.
- Cluster reduced to 4 hosts. The objects will continue to use the 2+1 scheme, spreading the stripe with parity across 3 hosts, while still having one spare fault domain to regain the prescribed level of resilience under subsequent host maintenance mode or failure conditions.
- Cluster reduced to 3 hosts. The objects will continue to use the 2+1 scheme, spreading the stripe with parity across 3 hosts. With a 3-host cluster, it can still suffer a host outage and maintain data availability but without the prescribed level of resilience.
Figure 4. Adaptive RAID-5 adjusting to the host count of the cluster.
While the animation above shows a cluster that is reducing the host count, the logic will also apply to a small cluster increasing host counts. vSAN also has protections in place to prevent capacity-constrained conditions from occurring because of a policy change.
Note that the 6-host cluster shown in Figure 4 could certainly use storage policies based on RAID-6, as that requires data and parity to be spread across 6 hosts. We generally recommend RAID-6 on clusters with 7 or more hosts so that if there was a failure of a host, another host would be available to regain the prescribed level of resilience.
Comparing the Space Efficiency and Host count Requirements
The following table will help you compare the space efficiency options between storage policies, as they relate to the ESA and OSA.
Figure 5. Comparing space efficiency and host count requirements in ESA (highlighted in green) and OSA.
Recommendation. For vSAN clusters running the OSA or ESA, strive to have a cluster host count with at least one more host than the minimum required by the storage policies used. This will allow for a storage policy to regain its prescribe level of resilience during a host failure or maintenance for a sustained period.
Summary
Erasure coding is not new to vSAN, but when using RAID-5 with the vSAN ESA, new levels of adaptability and space efficiency will be available to your workloads, all while delivering performance at the level of RAID-1 mirroring.