vSphere with Tanzu - Upgrade, Support and Kubernetes Versions

Last Update: 10-Jan-2024

Overview

vSphere with Tanzu is a vSphere capability that allows administrators to run modern workloads alongside the traditional Virtual Machines (VMs) on a vSphere Cluster. It delivers on-demand, production-ready Kubernetes as part of vSphere. vSphere with Tanzu achieves this with the help of two key components Supervisor and Tanzu Kubernetes Clusters.

Supervisor Cluster

When Workload Management is enabled on a vSphere Cluster two things happen. First, a Supervisor Cluster is created. This cluster is the Kubernetes control plane. Second, the ESXi hosts then become Kubernetes worker nodes capable of running pods directly.

Tanzu Kubernetes Clusters

In addition to running pods directly on the ESXi hosts, Supervisor allows administrators to deploy independent Kubernetes workload/guest clusters called Tanzu Kubernetes Clusters (TKC). These are virtual machines running the container based workloads. This is made possible by Tanzu Kubernetes Grid Service (TKGs) which is a set of controllers and custom resources deployed on the Supervisor to provide the lifecycle management of the TKCs.

Tanzu Kubernetes Releases

The Kubernetes distribution that powers vSphere with Tanzu is built using a common Tanzu Kubernetes Core. The Supervisor Cluster and TKC clusters use this common core. The Supervisor Cluster virtual machines are built using an opinionated installation of Kubernetes and components that integrate with vSphere.

Tanzu Kubernetes Clusters (TKC) are built using what’s referred to as a TKr. This is a combination of an operating system such as Photon or Ubuntu combined with Tanzu Kubernetes Core components necessary to provide Pod functionality. A TKr includes components such as Antrea and Contour that can be deployed in Tanzu Kubernetes Grid cluster. A TKr provides upstream aligned Kubernetes software distribution, signed, tested, and supported by VMware.

Although the source of Kubernetes distribution is the same for both Supervisor and TKCs, it is important to note the following

• Kubernetes distributions provisioned on Supervisor and Tanzu Kubernetes Cluster are independent of each other
• Kubernetes packaged in Supervisor is an opinionated installation of Kubernetes.
• Kubernetes packaged in Tanzu Kubernetes Clusters is an upstream aligned, fully conformant distribution of Kubernetes provided via TKrs.

Upstream Kubernetes Policy

Tanzu Kubernetes Release (Tkr) is aligned to the upstream Kubernetes. Therefore, the upstream Kubernetes policies impact the release cadence and the support policies for Kubernetes versions supported.
The following are key aspects of upstream Kubernetes policies relevant to the vSphere with Tanzu support policy-

• Upstream Kubernetes will follow three releases per year cadence.
• Upstream Kubernetes has a support policy of "N-2", which means at any point in time, the Kubernetes project maintains release branches for the most recent 3 minor versions.
• Kubernetes project policies require sequential upgrades; skipping minor versions is not supported.

TKr Support Policy

TKr will support (N-2) versions of Kubernetes (where N is the latest TKr released by VMware as listed in the release notes). This means TKr provides support for the three most recent versions of the Kubernetes released. Each Kubernetes version released by TKr is supported for a minimum of 12 months from the time VMware releases them (as opposed to when they were released upstream). In addition, refer to the compatibility matrix for minimum vCenter/vSphere versions required to run a TKr.  You can refer to the TKr Release Notes page for more information.

Supervisor Cluster Support

The support policy for the Supervisor Cluster is based on the TKr Support policy. As described in the above section, the Supervisor Cluster will support the 3 most recent versions of the Kubernetes versions released by TKr. If for any reason a specific deployment is on an unsupported version of the TKr , the first and foremost action required would be to move to a supported version, before VMware can provide any support. In addition to the latest version of the Kubernetes, every vCenter Release will package two previous releases of Kubernetes, supported by TKr the time of release.

Source:  Kubernetes Version Skew Policy, Kubernetes Release Cadence

Kubernetes Versions in Supervisor

Every vCenter release (major, update, or patch releases) will have three versions of Kubernetes in Supervisor. A newer version of Kubernetes may be released in any release of the vCenter (Major, Update or patch release). Availability of the new versions of Kubernetes in vSphere with Tanzu follows the release of the upstream Kubernetes version, and the timeline of release depends on the quantum of changes in the upstream Kubernetes release. Whenever Supervisor adds a newer Kubernetes version, the oldest version goes out of support. For example, here is a sample of vCenter releases. 

Supervisor Kubernetes included in vCenter Releases

You can find the full list of Kubernetes versions supported in vCenter below.

Sequential Upgrade Requirement

Upstream Kubernetes requires upgrades to be sequential, (i.e upgrade to the next minor version only). Since vSphere with Tanzu packages three Kubernetes versions, vSphere administrator will not have to upgrade to the immediate next version of vSphere with Tanzu, but will have a choice of upgrade paths, depending on the current version of vCenter. See Upgrade Path Rules in the below section

Support Policy

Although vSphere with Tanzu is part of vCenter, since it packages a distribution of upstream Kubernetes, the support policy of vSphere with Tanzu is similar to that of the upstream Kubernetes. 

Support for up to 3 most recent upstream Kubernetes release 

The latest update to the Kubernetes release cadence has been published in the official Kubernetes blog. As per this, Kubernetes will follow three releases per year cadence. Each released version is supported for a period of 12 months and a maintenance release of another 2 months. Kubernetes released with vSphere with Tanzu will also have the same support period as the corresponding upstream version. 

In addition to the latest Kubernetes version, the 2 previous releases which are in support are also packaged along with the release of vSphere with Tanzu.  Security Vulnerability and Bug fixes to upstream Kubernetes are made available in subsequent vCenter Patch Release as per the severity of the CVSS Score at the discretion of VMware. 

Upgrades

In the past, vSphere Administrators were required to upgrade to the latest vCenter releases very infrequently. They only had to consume the patch releases to keep the vCenter secure from Security Vulnerabilities and critical bugs. However, because of the speed of the upstream Kubernetes development, vSphere Administrators will now have to update their vCenter at least once every 9 - 12 months (approximately), in order to remain on the supported Kubernetes version. vSphere Administrators have to only update the vCenter (and not the ESXi hosts separately) in order to upgrade to the latest Kubernetes. vSphere with Tanzu is packaged with vCenter and the bits required to update the hosts are also packaged as part of this. The ESXi hosts need not be separately upgraded every time Supervisor is upgraded (provided the host ESXi versions are compatible with the upgraded vCenter version). The Supervisor upgrade process will update the bits in the ESXi hosts. The cluster updates do not require a reboot of the ESXi hosts. 

Deployments fallen out of support

vSphere with Tanzu will be able to provide support for those clusters which are on the supported versions of Kubernetes. If for any reason a specific deployment is on an unsupported version of Kubernetes, the first and foremost action required would be to move to a supported version before VMware can provide any support. 

Upgrade Path Rules

vSphere with Tanzu is packaged as part of vCenter releases. Because upstream Kubernetes requires Sequential upgrade, the upgrade path for vCenter depends on the included Kubernetes versions. If vCenter does not have any cluster with Supervisor enabled, then the limitation is not applicable. This section explains how customers can determine the upgrade paths available to them. 

Determining the Upgrade Path for vSphere with Tanzu

The following rules help determine the possible upgrade paths for vSphere with Tanzu upgrades.

Upgrade Path Rules

• Source and Target vCenter Releases have at least 1 overlapping version of Kubernetes packed in them (example 1)

OR

• The target version should contain the immediate next version of Kubernetes present in the source vCenter Release (see example 2 below)

If either of these rules is not satisfied, the vSphere administrator cannot upgrade to the Target release version.

Example

Consider the below example - assume the vSphere administrator intends to upgrade from VC Release X to one of the available releases.

Auto Upgrade

In example 2, while the upgrade is possible, vSphere with Tanzu performs an auto upgrade, upgrading any version on version 1.16 (unsupported versions) to the immediately supported version. 

Tanzu Kubernetes Cluster Support

The support policy of Tanzu Kubernetes Cluster is based on the TKr Support policy. As described in the above section, the TKC will support the 3 most recent versions of the Kubernetes versions released by TKr. If for any reason a specific deployment is on an unsupported version of the TKr , the first and foremost action required would be to move to a supported version, before VMware can provide any support.

Frequently Asked Questions

References

• https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/index.html
• https://kubernetes.io/blog/2021/07/20/new-kubernetes-release-cadence/
• https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-70CAF0BB-1722-4526-9CE7-D5C92C15D7D0.html
• https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-releases/services/rn/vmware-tanzu-kubernetes-releases-release-notes/index.html#about-vmware-tanzu-kubernetes-releases
• https://core.vmware.com/resource/vsphere-tanzu-highly-available-kuberneteshttps://core.vmware.com/resource/vsphere-tanzu-highly-available-kubernetes