Using the Perennially Reserved Flag for WSFC RDMs.
In some cases where customers are using pRDMs in their environment, host boot times or storage rescans can take longer than normal. The reason for these longer times is each LUN attached to a host is scanned at boot or during a storage rescan. Typically, RDMs are provisioned to VMs for Microsoft WSFC or other clustered applications and are not directly used by the host. During the scan, ESXi attempts to read the partitions on all the disks but it is unable to for devices persistently reserved by the WSFC. Subsequently, the longer it can take a host to boot or rescan storage devices. The WSFC uses SCSI-3 Persistent Reservation to control locking between the nodes of the WSFC which, blocks the ESXi hosts from being able to read them.
VMware recommends implementing perennial reservation for all ESXi hosts hosting VM nodes using RDMs. Check the following KB1016106 or more details.
The question then arises; How can you get the host to not scan these RDMs thus reducing boot or rescan times? I’m glad you asked!
There is a device flag called “Perennially Reserved” which tells the host the RDM should not be scanned and that device is used elsewhere (perennially) in the environment. Before vSphere 7, this flag can be enabled via CLI and requires the UUID (naa.ID).
The command to set the flag to true:esxcli storage core device setconfig -d <naa.id> --perennially-reserved=true
To verify the setting:esxcli storage core device list -d <naa.id>
In the device list you should see:Is Perennially Reserved: true
When setting this option, it must be run for each relevant RDM used by the WSFC and on every host with access to that RDM. You can also set the Perennially Reserved flag in Host Profiles.
With the release of vSphere 7, setting the Perennially Reserved flag to true was added to the UI under storage devices. There has also been a field added to show the current setting for the Perennially Reserved flag.
Once you select the RDM to be Perennially Reserved, you have the option to “Mark the selected device as perennially reserved” for the current host or multiple hosts in the cluster. This eliminates the manual process of setting the option per host via CLI. If preferred, you can still use ESXCLI, PowerCLI or Host Profiles.
Once you click YES, the flag will be set to true and the UI updated.
You may also unmark the device using the same process.
Short Clip showing process:
Setting the Perennially Reserved flag on your RDMs used by your WSFC is recommended in the clustering guides. When set, ESXi no longer tries to scan the devices and this can reduce boot and storage rescan times. I have added links below to resources on clustering guides and the use of this flag. Another benefit of flagging RDMs is you can easily see which devices are RDMs and which are not which can help avoid an RDM being accidentally used for other functions.
Important Note: You should only use the perennially reserved flag for RDMs or LUNs you do not what scanned. Standard Datastore LUNs should NOT be tagged with the flag. LUNs used for a datastore are controlled by ESXi and must be scanned at boot and accessible to the host, tagging them would disable the Datastore.
Resources:
- ESXi/ESX hosts with visibility to RDM LUNs being used by MSCS/WSFC nodes with RDMs may take a long time to start or during LUN rescan (1016106)
- Guidelines for Microsoft Clustering on vSphere (1037959)
- Microsoft Windows Server Failover Clustering (WSFC) with shared disks on VMware vSphere 7.x: Guidelines for supported configurations (79616)
- Microsoft Windows Server Failover Clustering (WSFC) with shared disks on VMware vSphere 6.x: Guidelines for supported configurations (2147661)