vSAN iSCSI Target Usage Guide
Native VMware Availability Options for vSAN
VMware vSAN™ iSCSI targets can be used to provide SCSI-3 locking, and LUNs required for a number of clustered applications. It should be noted there are many options for clustering applications.
VMware vSAN™ iSCSI targets can be used to provide external storage for a number of operating systems and applications. It should be noted when deciding on clustering an application that there are a number of choices for providing highly available services. There are also a number of alternatives both native to vSphere and to application layers that may be preferred, and not require the use of iSCSI connections to the vSAN iSCSI target.
vSphere High Availability™ (HA)
vSphere High Availability allows for virtual machines to fail over to other hosts in less than 10-minutes with a recovery point of zero. vSphere HA is supported on vSAN datastores.
VMware Fault Tolerance™ (FT)
VMware Fault Tolerance is a process where a virtual machine, called the primary vm, replicates changes to a secondary vm created on an ESXi host other than the one hosting the primary vm. Fault Tolerance can deliver both a recover point objective (RPO) as well as a Recovery Time Objective (RTO) of zero. Fault Tolerance is supported on VMware vSAN.
VMware vSphere® Replication™
VMware vSphere® Replication is a virtual machine data protection and disaster recovery solution. It is fully integrated with VMware vCenter Server™, providing host-based, asynchronous replication of virtual machines. When replicating from vSAN to vSAN vSphere Replication can deliver a 5-minute recover point objective.
Application Clustering Solutions
While it should be noted that vSphere High Availability, Fault Tolerance, and Replication can cover the vast majority of use cases in an easy to manage, low cost manner, there are a number of use cases for application level clustering. It should be noted that iSCSI will enable some new application clustering options, while others work today simply having VMDKs stored on VMware vSAN.
As of 7.0U2 and 6.7U3, Clustering with Redhat has been validated using the vSAN iSCSI service. Specifically, fence_mpath was validated (supports Multipathing).
Microsoft Always ON Availability Groups
Operate in a true shared-nothing fashion. They can be configured for either local synchronous replication or remote asynchronous replication. They are fully supported using native VMDK objects stored on vSAN storage.
Failover Clustering Instance (FCI)
FCI is supported on vSAN as of release 6.7, for more information see here . This includes all shared volumes across windows instances.
Microsoft File Services
Microsoft has a number of of availability solutions for file services. Built into their server operating system.
Distributed File System Replication DFS-R
The Distributed File System Replication (DFS-R) service replication engine that can keep folders synchronized using compression and differential comparison. DFS-R works in a shared nothing environment and is fully supported using native VMDK objects stored on VMware vSAN storage.
File Server Failover clustering
2008R2 2012R2 Failover Cluster
Traditional File Server clustering in windows server requires SCSI-3 locking, and shared storage. This is supported on VMware vSAN as of release 6.7 see here for details.
2012/2012R2 SoFS Cluster
Traditional File Server clustering in windows server requires SCSI-3 locking, and shared storage. While SoFS has advantages over traditional failover clustering there are limitations on what is recommended or supported in using SoFS vs. traditional failover clusters.
Windows 2016 Storage Replica (SR)
Storage Replicas are agnostic to storage hardware and has no specific storage hardware requirements. It is supported and will run on top of VMware vSAN.
Exchange as of Exchange 2010 no longer uses shared volumes for clustering. Database Availability Groups (DAG) use a shared nothing design, and is fully supported using native VMDK objects stored on a vSAN datastore.
Virtualized Oracle on vSAN is fully supported using native VMDK objects stored on a vSAN datastore. Note, you will need to use the multi-writer flag and there are a number of considerations (no HotExtend, VADP, CBT support). See KB 2121181 for more information.
Physical servers running Oracle RAC are supported with VMware vSAN using the iSCSI Target Service. Support is being extended specifically for Oracle RAC 12c, 11gR2.
Third party solutions
3rd party solutions also offer a variety of replication solutions using both VMware vSphere Storage APIs – Data Protection (Formerly known as VADP) as well as using vSphere APIs for I/O Filtering (VAIO).
VMware guidance: vSphere High availability offers an excellent RPO and RTO for most workloads, and vSphere replication. When lower recovery times are needed, should be application level clustering often no longer requires clustered volumes.
iSCSI, like any storage protocol, requires proper security considerations.
iSCSI storage traffic is transmitted in an unencrypted format across the LAN. Therefore, it is considered best practice to use iSCSI on trusted networks only and to isolate the traffic on separate physical switches or to leverage a private VLAN. All iSCSI-array vendors agree that it is good practice to isolate iSCSI traffic for security reasons. This would mean isolating the iSCSI traffic on its own separate physical switches or leveraging a dedicated VLAN (IEEE 802.1Q).
Encryption and Authentication
CHAP (Challenge Handshake Authentication Protocol) verifies identity using a hashed transmission. The target initiates the challenge. Both parties know the secret key. It periodically repeats the challenge to guard against replay attacks. CHAP is supported by the vSAN iSCSI Target service. bidirectional CHAP is supported.
Availability and Performance Best Practices
Best practices for scaling availability and performance.
KB57344 documents best practices for the iSCSI target service.
Availability Best Practices
Multi-Path IO (MPIO) is supported by the vSAN iSCSI Target Service. Every target has an owner and initial connections will be redirected using iSCSI redirects to the owner's path. In the event of a failure, reconnection attempts will be redirected to the new iSCSI LUN target owner. An initiator can connect to any host, but will always be redirected to the current active host.
Performance Best Practices
iSCSI Targets have a limited maximum queue depth and it is recommended to utilize more targets to increase performance. It should be noted that a given target will only be active for a single host so deploying more targets will lead to an evener usage of paths for performance balancing. You can see the I/O owning Host from within the UI. It should be noted that iSCSI utilizes more compute overhead, and because of added pathing will add additional latency and overhead compared to running Virtual Machine disks directly on the VSAN datastore. If performance is a concern, iSCSI should only be used when native VSAN is not an option.
Interoperability and Support Considerations
Discussions on supported configurations.
vSAN policies and capabilities
As the iSCSI LUNs are stored as VMDK’s SPBM (Storage Policy Based Management) can be used to manage the characteristics of the LUNs. Space efficiency features including RAID-5 and Deduplication and Compression are fully supported.
vSphere and VMware feature support
ESXi host support
Use of the Virutal SAN iSCSI Target for providing storage directly to vSphere is not currently supported. The intent of this feature is to provide for extended use cases where application clustering requires it, as well as physical workloads outside of the VSAN cluster.
vSphere and vSAN features not currently supported
The following features are not supported by iSCSI Target service volumes.
- vSphere Replication
- vSphere Data Protection API’s
- vSAN Stretched Clusters
- HCI Mesh
For data protection, in guest tools, agents, or application level data replication tools will need to be leveraged.
vSAN 7 Update 1 - iSCSI Stretched clustering Support
vSAN 7 Update 1 introduces iSCSI Support for Stretched Clusters. This includes affinity for the Preferred or Secondary site.
This Ensures that iSCSI traffic for initiator/target does not cross sites
Proper failover to remote site if a fault condition is detected
Operational note: For VMs, storage policies should also be used that specify the affinity preference, and if the iSCSI initiator is coming from a VM using the vSAN iSCSI service, then a DRS affinity rule should be created to ensure proper initiator/target/object affinity.
Supported Operating Systems
The following operating systems are supported.
Windows server 2019, 2016 , 2012 R2, 2012, 2008 R2, 2008
RHEL 8, RHEL 7, RHEL 6, RHEL 5
SUSE® Linux Enterprise Server 12, SLES 11 SP4/SP3/SP1
Oracle Linux 7, Oracle Linux 6