vSphere Lifecycle Manager

Introduction

vSphere and vSAN 7 introduced an entirely new solution for unified software and firmware management that is native to vSphere. vSphere Lifecycle Manager (vLCM) is the next-generation replacement to vSphere Update Manager (VUM), and is built on desired-state, or declarative model that provides lifecycle management for the hypervisor and the full stack of drivers and firmware for the servers powering your data center.

Typical challenges with server lifecycle management

Healthy servers are essential for optimal data center operations. This is why VMware and server vendors work closely together to ensure servers on the VMware Compatibility Guide have been validated and are certified to support your workloads. But having certified hardware is only half the equation because server health relates to hardware and software. Server administrators need a lifecycle management plan for software, firmware and drivers. This plan might consist of disparate tools and varying update cycles. In addition, server maintenance is especially time-consuming as organizations increase in size. The potential for disruption and the complexity around the process can discourage organizations from implementing regular lifecycle management.

vSphere Update Manager

VMware’s long-standing solution for upgrade and patch management of the hypervisor is called “vSphere Update Manager” (VUM). VUM is a good way for an administrator to manage the upgrade process of the hypervisor with the various hosts living in the data center. Its focus is primarily on upgrade and patch management of the hypervisor residing on the host. While it improves host management, there is no firmware management integration. As a result, administrators have to manage server firmware (i.e. BIOS, storage IO controllers, disk devices, NICs, and remote management tools) separately. While VUM was a step in the right direction, it lacked full server-stack management.

Unified software and firmware lifecycle management with vLCM

vSphere Lifecycle Manager, introduced in vSphere 7, introduces unified software and firmware lifecycle operations for vSAN clusters, which includes the base ESXi version, drivers, and the full server-stack firmware. This new capability addresses a long-standing customer pain point of having to separately manage the full server-stack firmware via vendor products such as HPE SUM/OneView or Dell EMC iDRAC/OpenManage. vLCM also manages at the cluster level ensuring consistency of software, firmware and drivers across all hosts in the cluster.

Architecture

Several components makeup vSphere Lifecycle Manager and work together to deliver the vSphere Lifecycle Manager functionality and coordinate the major lifecycle management operations that it provides for. vSphere Lifecycle Manager is a service that runs in vCenter Server. No additional installation is required to start using that feature. Some of the components that make up vSphere Lifecycle Manager run in the vCenter Server appliance as part of the respective service, while others are external to vCenter Server.

Desired State Model

vLCM is based on a desired state or declarative model (similar to Kubernetes) that allows the user to define a desired image (ESXi version, drivers, firmware) and apply to an entire vSphere cluster. Once defined and applied, all hosts in the cluster will be imaged with the desired state. Managing at the cluster level is a superior model compared to individual hosts as it provides consistency and simplicity.

A vLCM desired state image consists of the following elements:

ESXI Version (Required)
Vendor Addon (optional)
Firmware and Drivers Addon (optional)
Individual Components (optional)

ESXi Version: The base image is an ESXi release version that contains an image of VMware ESXi Server and additional components such as drivers and adapters that are necessary to bring up a server. This is the only required element of a vLCM image.

Vendor Addon: A vendor add-on is a collection of software components for the ESXi hosts that OEMs create and distribute. This vendor add-on can contain drivers, patches, and solutions. The add-on is a collection of components that does not represent a complete, bootable image. You cannot use vendor add-ons on their own. To customize an ESXi release, you must add a vendor add-on to an ESXi base image. The vendor addon when combined with an ESXi base image is the equivalent to a server vendor custom ISO. When applied, the add-on can add, update, or remove components that are part of the base image.

To examine the contents of a vendor addon, open Lifecycle Manager and from the Image Depot tab highlight the respective vendor addon.

Firmware and Drivers Addon: The firmware and drivers add-on is a special type of vendor add-on referred to as a Hardware Support Package (HSP) designed to assist in the firmware update process. It contains firmware for a specific server type and corresponding drivers. The content in a HSP is unknown to vLCM and completely handled by the Hardware Support Manager (HSM). This is in contrast to the ESXi version and Vendor Addons which are stored in Update Manager’s internal depot (either downloaded directly from VMware for on-line systems or manually uploaded by admins for air-gapped ("dark site") systems.

Component: A component is the smallest unit that can be included in a desired state image. For example, a driver is a component.

Drift Detection

One of the key benefits of using a desired state model compared to individual host management solutions is drift detection. When an image is defined, vLCM will verify the ESXi and vendor firmware and drivers have the specified versions. Any added (or missing) component is considered drift and identified as noncompliant. Administrators will be notified of any drift and from the applied image and can easily remediate. Compliance checks can be run on a cluster at any time. This is a non-destructive process that doesn’t require maintenance mode and can ensure that all hosts in a cluster are exactly as specified in the desired image.

vSphere Lifecycle Manager

In vSphere 7.0, vSphere Lifecycle Manager encompasses the functionality that Update Manager provided in earlier vSphere releases and enhances it by adding new features and possibilities for ESXi lifecycle management at a cluster level.

vSphere Lifecycle Manager is a service that runs in vCenter Server. Upon deploying the vCenter Server appliance, the vSphere Lifecycle Manager user interface becomes automatically enabled in the HTML5-based vSphere Client.

vSphere Lifecycle Manager can work in an environment that has access to the Internet, directly or through a proxy server. It can also work in a secured network without access to the Internet. In such cases, you use the Update Manager Download Service (UMDS) to download updates to the vSphere Lifecycle Manager depot, or you import them manually.

Hardware Support Manager (HSM)

ESXi and vendor addons are managed in Lifecycle Manager Image Depot. To provide full server-stack firmware management capabilities with vLCM, server vendors provide a vCenter plug-in called a Hardware Support Manager (HSM). Customers must separately purchase, install and manage the HSM with guidance and support from the server vendor.

Server Vendor	Hardware Support Manager
Dell	Open Manage Integration for VMware vSphere OMIVV
HPE	HPE iLO Amplifier Pack
Lenovo	Lenovo xClarity Integrator (LXCI)
Hitachi	Hitachi Unified Compute Platform Advisor

Hardware Compatibility Checks

Maintaining compatible hardware on servers supporting production workloads is critically important. vSphere/vSAN 7 introduced a few enhancements to hardware compatibility, verifying whether the physical I/O device controllers are compatible with the ESXi version specified in the vLCM image. Now in vSphere/vSAN 7 Update 3, vLCM has been enhanced to support the validation and update of firmware for storage devices as well. The integrated hardware compatibility checks will scan the desired-state image and notify you of any incompatibilities in your environment before every remediation. Compatibility is checked for vSAN storage controllers and storage devices. If a device is incompatible with the ESXi version in the desired image, an error will appear in the Hardware Compatibility tab of Updates.

When you initiate a hardware compatibility check for a cluster, vSphere Lifecycle Manager verifies that the components (software, drivers and firmware) in the image are compatible with all storage controllers, and storage devices on the hosts in the cluster as per the vSAN VCG. The task also checks whether the driver and firmware versions specified in the image are compatible with the cluster hardware as per the vSAN VCG. KB60382 identifies all IO Controllers that vSAN supports firmware updating.

Hardware compatibility issues are reported as warnings, but they do not prevent you from remediating the hosts in the cluster against the image. Hardware compatibility checks are available only for vSAN clusters that are managed with a single image (vLCM). At the host level, hardware compatibility checks can be run against the VMware compatibility Guide as well as the vSAN HCG for running state. Host level hardware compatibility checks do not have access to the firmware information in the HSMs and do not check against the vLCM cluster level desired image.

Reference Host Seeding

The workflows used by administrators for both new cluster creation, as well as a greenfield environment where a vCenter Server appliance is bootstrapped onto a single host have been updated to accommodate the ability to easily reference a host for easy compliance. vLCM will extract the image from a single host and apply to other hosts in the cluster in a seamless fashion, saving time and effort. You can also import an image from an ESXi host that is in the same or a different vCenter Server instance. You can also import an image from an ESXi host that is not managed by vCenter, move the reference host to the cluster, or use the image on the host and seed it to the new cluster without moving the host.

Intelligent Update Recommendations

vLCM update recommendations auto-regenerate after certain change events such as a refreshing of the vSphere depot or modification of the desired cluster image. Updates include ESXi versions (patch or upgrades) and vendor-addons (HSPs) – if an HSM is configured for the cluster. If no HSM is configured, then only software updates will be provided (ESXi or drivers). In addition, the administrator will be notified of new or regenerated update recommendations through vCenter events and alarms.

Faster upgrades with Quick Boot suspend to memory

Host updates are a necessary task in the data center. When a host is contributing CPU, memory, and storage resources, the more quickly a host can be updated the better. Less time offline means more time in production. This is where vSphere Quick Boot comes into play, where host restarts during an upgrade can be accelerated by bypassing the hardware and firmware initialization process of a server.

vSAN 7 U2 provides better integration and coordination for hosts using Quick Boot to speed up the host update process. By suspending the VMs to memory, and better integration with the Quick Boot workflow, the amount of data moved during a rolling upgrade is drastically reduced due to reduced VM migrations, and a smaller amount of resynchronization data. When the circumstances are suitable, Quick boot can deliver a much more efficient host update process if the workloads are able to be suspended for a brief period of time.

Witness Host Appliance support for Stretched and 2-Node clusters

vSphere Lifecycle Manager (vLCM) has proven to be a major step forward in providing a framework for desired state lifecycle management of vSphere hosts in a cluster. While vLCM works with both vSAN stretched cluster and vSAN 2-node topologies, it was unable to manage the lifecycle of the witness host appliance – a key component of these topologies. This was in part because vLCM could only manage hosts within a cluster, and witness host appliances had a strict requirement to reside outside of a vSphere cluster. vSAN 7 U3 now provides support for managing the lifecycle of the vSAN witness host appliance for vSAN stretched cluster and 2-node topologies. Once a stretched cluster or 2-node cluster that meets the criteria is managed by vLCM, the witness host appliance will also be managed by vLCM. Hosts and witness appliances will be updated in the recommended order to maintain availability.

A few considerations:

The witness host must be a virtual appliance.
The witness host appliance must be managed by the same vCenter Server as the cluster being upgraded.
A shared witness cannot be part of a vLCM cluster image since they can potentially manage hosts in multiple clusters.
This feature is applicable to versions equal or newer than vSAN 7 U3

vLCM on VMware Cloud Foundation

Previous versions of Cloud Foundation have simplified lifecycle management operations by enabling patching and updating of the core components of the VMware software stack at the Workload Domain level. VMware Cloud Foundation 4 introduces the integration of vSphere Lifecycle Manager (vLCM). vLCM complements the capabilities already found in Cloud Foundation’s SDDC Manager by offering integration deeper into the stack with firmware management for qualified vSAN ReadyNodes. vLCM will automatically validate the HBA firmware is on the hardware compatibility list, giving admins confidence that their updates will be successful before applying updates to match the defined desired state. Integration with vLCM will further simplify lifecycle management of the full stack, including vSAN ReadyNode hardware, through Cloud Foundation.

VMware Cloud Foundation supports vLCM with the following considerations:

Workload domains are supported with vLCM, but management domains must still use vSphere Update Manager (VUM).
When provisioning new workload domains you choose either Kubernetes or vLCM. vLCM cannot manage a Kubernetes workload domain in VCF 4.
NSX-T and vLCM are supported for workload domains. NSX-T with vLCM is not supported outside of VCF.

Additional Information

vLCM Capable ReadyNodes

Before taking advantage of the full server-stack firmware management capabilities of vLCM, it’s important to verify that your servers are listed as “vLCM Capable ReadyNodes”. To verify this, select “vLCM Capable ReadyNode” in the vSAN ReadyNode Additional Features section of the VMware Compatibility Guide.

vSphere Update Manager vs vSphere Lifecycle Manager

The default lifecycle management configuration for vSphere 7 is VUM. To switch to using vLCM click the “MANAGE WITH A SINGE IMAGE”. Managing a host with vLCM has the following prerequisites:

Hosts must be running ESXi 7.0 or higher
All hosts in the cluster must be from the same vendor
Hosts may not be stateless

Since the identical "desired state" will be applied to all hosts, the hosts must be compatible with a single image.

Clusters must be managed by either VUM or vLCM, but not both. Since vLCM is managed at a cluster level, it is possible to manage some clusters with vLCM and others with VUM. This is common for organizations that have a mix of some servers that are vLCM Capable ReadyNodes and others that are not.