Using the Offline Bundle Transfer Utility for Disconnected VMware Cloud Foundation Sites

Disconnected Sites

VMware Cloud Foundation (VCF) offers fully orchestrated lifecycle management for your VMware SDDC infrastructure. To update components such as vCenter Server, ESXi, and NSX-T, SDDC Manager must download software bundles from an online depot. If your SDDC Manager has Internet connectivity, simply enter your Customer Connect credentials and select the bundles to download.

However, what if your VCF deployments cannot access the Internet? Some organizations restrict automatic online updates due to security policies or compliance requirements. These ‘disconnected’ or ‘dark’ sites (sites without Internet connectivity) must use an alternative method to bring updated software bundles into the VCF environment.

Offline Bundle Transfer Utility

For disconnected VCF deployments, customers must use the Offline Bundle Transfer Utility (OBTU) for lifecycle management. It is not included with SDDC Manager by default, so you must download the tool from Broadcom Support Portal.

The OBTU is used in two complementary ways:

On a machine with Internet access, an administrator can download required bundles and then copy them to SDDC Manager
On the SDDC Manager itself, OBTU is used to import the bundle files so they can be used to update the infrastructure

The rest of this article will walk you through the workflow so you can understand how to use the OBTU. You can also refer to the product documentation for additional information.

Downloading Metadata and Bundles

First you will need a Linux or Windows machine with Internet access, a Java runtime, and plenty of disk space. Depending on the versions of VCF in your specific environment, downloaded bundles can consume 100GB or more. If your disconnected VCF deployment is truly air-gapped, then you would need to use a portable storage device to physically transport the bundles. Otherwise, if you have the ability to connect to the SDDC Manager, you can copy the bundles directly after downloading.

In the companion demo video, you can see how to configure a Linux system for use with OBTU. Below are the commands you can use to check for available disk space, verify the checksum of the OBTU download, prepare the directories, and extract the utility.

df -h /var/opt
shasum lcm-tools-prod-23391199.tar.gz
sudo mkdir /var/opt/obtu
sudo chown $USER:$USER /var/opt/obtu
sudo mkdir /opt/obtu
sudo chown $USER:$USER /opt/obtu/ 
tar zxvf lcm-tools-prod-23391199.tar.gz --directory=/opt/obtu/
cd /opt/obtu/bin/
chmod +x lcm-bundle-transfer-util

Once the tool is ready for use, it is highly recommended that you start a tmux session so that any disruptions to your SSH connection will not terminate the long-running download process. If you’re not familiar with tmux, it is similar to the older ‘screen’ utility and it allows you to disconnect your session from a remote system without logging out - among other things.

In addition to the bundle download, there are several metadata files that must also be fetched and imported into SDDC Manager. These are the manifest, compatibility data, and vSAN HCL.

After the downloads complete, you need to transfer the entire directory structure to the SDDC Manager VM that is part of your disconnected VCF deployment. If you have direct access, rsync is the best way to do this. Otherwise, you might need to copy the files to portable media or to another intermediate jump box that does have access to the disconnected VCF. There is an existing directory on the SDDC Manager VM that is recommended for use, since it has ample space for bundles, so copy your downloaded bundle directory to /nfs/vmware/vcf/nfs-mount/.

The following example commands show how to download the metadata and software bundles for a VCF 5.1 to 5.1.1 upgrade. Adjust the parameters for your environment accordingly. The depotUser parameter needs to be the email address you have registered with Customer Connect - the tool will prompt for your password.

tmux

./lcm-bundle-transfer-util --download --manifestDownload \
--depotUser 'user@example.com' \
--outputDirectory /var/opt/obtu 

./lcm-bundle-transfer-util --download --compatibilityMatrix \
--depotUser 'user@example.com' \
--outputDirectory /var/opt/obtu
    
./lcm-bundle-transfer-util --vsanHclDownload  \
--outputDirectory /var/opt/obtu

./lcm-bundle-transfer-util --download \
--depotUser 'user@example.com' \
--outputDirectory /var/opt/obtu \
--sourceVersion 5.1.0.0

rsync -aP /var/opt/obtu vcf@sddc-manager:/nfs/vmware/vcf/nfs-mount/

The three metadata files will be arranged as follows after downloading:

Transferring and Uploading to SDDC Manager

After the software bundles and metadata files have been transferred to the SDDC Manager, you use OBTU to upload them for use. As with the system used for downloading, OBTU must first be installed. The commands below outline a suggested procedure to create a new directory, set ownership to the “vcf” user, and extract the archive file.

su -
mkdir /opt/vmware/vcf/lcm/lcm-tools
chown -R vcf:vcf /opt/vmware/vcf/lcm/lcm-tools
exit

tar zxvf lcm-tools-prod-23391199.tar.gz --directory=/opt/vmware/vcf/lcm/lcm-tools
cd /opt/vmware/vcf/lcm/lcm-tools/bin
chmod +x lcm-bundle-transfer-util

Now that the tool is executable, you can use it to upload the metadata and bundles. Start with the manifest, compatibility matrix, and vSAN HCL. Notice the inputDirectory parameter for the first two points to the directory that you copied to SDDC Manager but the HCL requires the full path to the JSON file.

Just as discussed above, use a tmux session for the long-running bundle upload process in case your SSH connection is disrupted. The upload process takes quite some time because each bundle is individually validated to ensure the integrity of the software.

tmux

./lcm-bundle-transfer-util --update \
--sddcMgrFqdn 'sddc-manager.vcf.sddc.lab' \
--sddcMgrUser 'administrator@vsphere.local' \
--sourceManifestDirectory /nfs/vmware/vcf/nfs-mount/obtu 

./lcm-bundle-transfer-util --update \
--sddcMgrFqdn 'sddc-manager.vcf.sddc.lab' \
--sddcMgrUser 'administrator@vsphere.local' \
--compatibilityMatrix \
--inputDirectory /nfs/vmware/vcf/nfs-mount/obtu

./lcm-bundle-transfer-util --vsanHclUpload \
--sddcMgrFqdn 'sddc-manager.vcf.sddc.lab' \
--sddcMgrUser 'administrator@vsphere.local' \
--inputDirectory /nfs/vmware/vcf/nfs-mount/obtu/vsan/hcl/all.json   

./lcm-bundle-transfer-util --upload \
--bundleDirectory /nfs/vmware/vcf/nfs-mount/obtu

Full Demo Video

The full process described above can also be seen in this short demo video:

Perform Infrastructure Updates as Usual

Once the bundles have been uploaded to VCF, you can proceed with infrastructure updates as usual by using the SDDC Manager web UI. For more information on software bundles and VCF lifecycle management, please take a look at this playlist.