Comparing the Original Storage Architecture to the vSAN 8 Express Storage Architecture

VMware vSAN 8™, Introduces the revolutionary Express Storage Architecture™.  This is an optional, alternative storage architecture to the vSAN original storage architecture also found in vSAN 8.  When running on qualified hardware in approved vSAN ReadyNodes, the vSAN Express storage architecture will offer supreme levels of performance, scalability, resilience, and data services without compromising performance.  The vSAN Express Storage Architecture unlocks the capabilities of modern hardware to allow the workloads of today and tomorrow perform as the hardware allows.

How is it different?

How does the vSAN Express Storage Architecture compare with the classic storage architecture of vSAN?  At a high level, the original storage architecture of vSAN was a two-tier architecture designed to accommodate a wide ranging set of older storage devices, while the vSAN Express Storage Architecture in vSAN 8 is a single-tier architecture optimized for high performance NVMe based TLC flash devices for both on-premises environments, and for the hyperscale public clouds.  With vSAN 8, VMware gives you the opportunity to select which architecture best suites your existing hardware, while allowing you to deploy new clusters with unprecedented levels of performance.  While the vSAN Express Storage Architecture is the future of vSAN, VMware customers should take great comfort that the original storage architecture of vSAN can be used side by side with the ESA, and will be supported for many years to come.

Performance without compromise

Previously with the vSAN original storage architecture, careful planning had to be made when deciding between using RAID 1, RAID 5 or RAID 6. Workloads needing to maximize capacity would chose RAID 5/6 while workloads needing maximum write performance would chose RAID 1. vSAN Express Storage Architecture uses a new log structured file system that allows the cluster to store data using RAID-6 at the performance of RAID-1. Compression has been changed to a per virtual machine setting, and will compress data before it traverses the network allowing increased throughput with lower networking overhead.     vSAN ESA uses a new native snapshot engine that is highly scalable and efficient.

Capacity Improvements

vSAN Express Storage Architecture allows the cluster to store data using RAID-6 at the performance of RAID-1. RAID 5 can now operate in a 2+1 or a 4+1 configuration. This brings RAID 5 support to clusters as small as 3 nodes while enabling a more capacity efficient stripe size on larger clusters.  vSAN 8 ESA removes the need for dedicated cache devices. The new log structured file system and IO path optimizations to the write path have been made to further reduce write amplification and reduce write latency. Removing the  disk groups concept has the added benefit of drives failures are now limited in scope to a single storage device. Without cache devices, the capacity of all devices is usable by the vSAN datastore. Combining the ability to use RAID 5/6 at all times, and the removal of the need for cache devices can significantly reduce the cost per GB for VMware vSAN clusters. new techniques are in place to improve the potential data reduction of the compression feature to as high as an 8:1 compression ratio for every 4KB block written, which is a 4x improvement from the original storage architecture. These new capacity savings are implemented in such a way to both save money without compromising on performance.

Security Improvements

vSAN Express Storage Architecture moves the vSAN Encryption process higher in the stack to occur on the host where the virtual machine resides. Data only needs to be encrypted once at rest, there is no longer a need to potentially decrypt and re-encrypt as was required when data would need to be decrypted to perform compression when moving from cache to capacity devices in the original storage architecture. This change minimizes CPU cost for encryption, as well as lowering of I/O Amplification to use encryption. Data encryption can be layered on top of the vSAN datastore encryption for added security.

Snapshots differences  

The snapshot architecture used in previous editions of vSAN can be thought of as an enhanced form of a redo-log based snapshot architecture that has been a part of vSphere for many years. Previously, vSAN 6 introduced the “vsanSparse” snapshots to alleviate some of the technical implications of traditional, redo-log based snapshots. This improvement did not resolve all challenges in regards to slow snapshot consolidation times, and performance degradation during snapshot operations or even long term retention. vSAN ESA uses an entirely new native snapshot system enables faster snapshot operation times (100x faster consolidation) and opens up potential new use cases. These new snapshots are accessible by backup APIs, making it a drop in replacement when using Virutal machine backup and replication products.

Disk Groups vs. Pools

vSAN 8 ESA removes the need for dedicated cache devices. The new log structured file system and IO path optimizations to the write path have been made to further reduce write amplification and reduce write latency. This has the added benefit of a failure of a given storage device has no impact on the remaining storage devices in the host. Without cache devices, the capacity of all devices is usable by the vSAN datastore.

Hardware Choices

Why the differences requirements?

A 4 device minimum was chosen at this time to enforce a minimum expected performance outcome as well as to enhance availability.

Highly performant, and durable TLC NVMe devices were chosen for the initial ESA ReadyNodes. They were chosen for their ability to deliver consistent performance, low latency and reduced CPU needed for storage processing.  To take full advantage of the near device-level performance of these ReadyNodes, 25Gbps will be a minimum requirement for host networking, but existing workloads moved to a vSAN ESA cluster will likely use less networking than on a vSAN cluster using the OSA.  For now at GA only ReadyNodes are supported, so a certain number of design decisions will effectively be made as part of the ReadyNode sizer’s selection limitations simplifying the time for vSAN architectural design.

More answers frequently asked questions on the vSAN ESA can be found on the vSAN FAQs dedicated to ESA related questions.