Platform Services / vCenter Server Upgrade
Server via the CLI Installer
The first topic we will cover in our Automating your vSphere Upgrade series is updating our core infrastructure. Upgrading Platform Services Controller(PSC) and vCenter Server via the CLI installer.
In our workshops one of the common things I hear is that most folks aren’t using the CLI based tools that have been included in the installer. I am a huge fan of these as they make migrations and upgrades seamless–especially when having to perform the task if you have multiple PSC’s and vCenter Servers. Using the CLI tools don’t have to be intimidating, as we go through this blog we will cover some how-to’s and benefits of using the CLI based approach.
When we start our vSphere Upgrade the first thing we want to do is to upgrade our PSC and vCenter Server before updating other components. If we happen to be using an Embedded PSC deployment the upgrade is simple, we can do the upgrade in one step (Step 1 and Step 2 combined). However, if we are using an External PSC deployment we must upgrade all of our PSC’s within the same SSO domain first prior to upgrading any vCenter Servers–so this becomes a multi-step process.
For this upgrade scenario we will be performing an upgrade from an
External PSC deployment using appliances on vSphere 6.0 to an External PSC deployment using appliances on vSphere 6.7.
Prepare Your JSON Configuration File for CLI Upgrade
When doing a CLI based upgrade, we use a configuration file template (JSON) that is part of the vCenter Server Appliance (VCSA) ISO. When you mount the ISO you will see a directory labelled vcsa-cli-installer and within that folder is a directory called templates. Once you are here you have the option to see the CLI templates for install, migrate and upgrade. As a reminder a migration is when we are moving from a vCenter Server on Windows, and an upgrade is when we are moving from VCSA to VCSA.
Since we are doing an upgrade we will browse to that folder and we are left seeing a vcsa6.0 and vcsa6.5 directory. The information we need when coming from each version is different so there is a unique set of templates for each version. Since we are upgrading from a vSphere 6.0 deployment we will head into the vcsa6.0 directory.
Once we are in this directory we can see a few different templates. We have a template for an embedded deployment, external PSC and external vCenter Server. Each template has an associated version whether or not we will be connecting to a vCenter Server or an ESXi host directly. The embedded template is for an embedded deployment which does both PSC and vCenter Server at the same time as we mentioned above, and the PSC and VCSA are separate templates as we need to do these one at a time.
Since we have an External PSC deployment and we are using a vCenter Server, we will first open the PSC_on_VC.json.
When opening this file at first glance you might say WOAH! That looks complicated, but there is quite a bit of extra text explaining each field. Lets fill in the blanks with the required information and take another look.
This looks a lot better, we can clearly see the details it is asking for. The vCenter Server to deploy the new appliance to, the appliance information, the temporary IP to use, and then last but not least the source information for the appliance we are upgrading. If we have a deployment where there are multiple PSC’s we could see how easy it would be to make a copy of this file, and edit lines 17 and 41 putting in the second PSC and then re-saving. How much time saving could this be!? With vSphere 6.5 allowing 10 PSC’s per SSO domain and vSphere
6.7 allowing 15 this could be quite a bit of time saved!
Next up, its time to upgrade our vCenter Server. To get our JSON file ready we will head back and grab the vCSA_on_VC.json template. Also remember that if you happen to have multiple VCSA’s to upgrade you can easily update lines 16 ,17 and 40 to upgrade other appliances in your environment.
Certain sensitive fields such as passwords can be left blank if chosen. When the CLI installer is run it will prompt for the password during runtime.
Now that we have prepared our templates for our upgrade, the next section we will jump into how to actually perform our upgrade using the CLI.
More information on preparing the JSON template can be found here.
Upgrade a vCenter Server Appliance or
Platform Services Controller Appliance by
Using the CLI
CLI templates in hand, its now for the fun. Using these templates we will automate our upgrade. vSphere 6.0 originally introduced this concept of a CLI based tool, however its been enhanced with vSphere 6.7. With vSphere 6.7 we now have a “batch template” technology built into the installer. Prior to vSphere 6.7 we had to run the installer against a single JSON file and wait for it to finish, we would then have to manually kick off the second JSON file when the previous one completes. With the batch upgrade we can put multiple JSON files into a single directory and point the installer to the directory and it is smart enough to map out the dependencies and upgrade the PSC’s first and then the vCenter Servers.
Before we actually perform our upgrade, I want to explain some of the benefits of the CLI installer. If we happened to be using the GUI based upgrade, everything in that upgrade would have to be manually entered into the GUI and obviously that is susceptible to typos and errors. Also, if we happened to enter some incorrect information and had to retry our upgrade we would again have to re-type all that information. What if we had a way to perform an upgrade test without ever touching our production environment? With the CLI Installer we do! And its called a pre-check. When running the CLI installer with the –precheck-only flag we have the ability to go through a full dry run of our upgrade. If we happened to have a typo, insufficient capacity or any other errors it would spit this out, and we could easily correct the settings.
To run our CLI based installer we will mount our vSphere 6.7 media as that is the version we are going to. We will again navigate to the vcsacli-installer directory. From here depending on the operating system of the machine we are on we will either navigate to the lin64, win32 or mac folder to use the supported installer for that operating system.
Here we will see quite a few files, but the one we want to focus on is vcsa-deploy.exe in the win32 folder since I will be running this from a windows machine. If we run .\vcsa-deploy.exe upgrade it will give us the details on how to properly use the CLI tool.
As I mentioned previously one of the benefits of using the CLI tool is that you can execute a pre-check of your template against the environment to make sure things will pass. Usually with upgrades and migrations a big issue we see with them failing is due to a smaller appliance size being chosen for the new appliance or insufficient capacity in the vCenter Server environment. In this environment we will execute our vcsa-deploy utilizing the pre-check option and see our results.
To run the pre-check I ran the following command: .\vcsa-deploy.exe upgrade –no-ssl-certificate-verification –accept-eula –acknowledgeceip –precheck-only C:\scripts\6.7\upg60to67\
I chose to point this to the folder to take advantage of the batch upgrade vs pointing it to the individual file, as I mentioned earlier it is smart enough to map out the dependencies as we can see from the below image.
Once the pre-check runs through we will either get a success or failure. In the image below my vCenter Server services were not running so the pre-check failed.
Now that we have the service running and re-run our pre-check we can see everything is successful and we can now proceed to remove the – precheck-only flag from our command.
.\vcsa-deploy.exe upgrade –no-ssl-certificate-verification –accepteula –acknowledge-ceip C:\scripts\6.7\upg60to67\
The upgrade process is now happening without any additional user input. If you need to step away for lunch or if a co-worker needs you there is no additional action that is needed until the end. If you were using the GUI based approach once Stage 1 where the appliance is deployed is complete you have to manually start Stage 2 to configure the upgraded appliance. With the CLI based installer this is not needed. When our upgrade is complete we will be presented with the vCenter Server and login information as we can see below.
More information on Upgrading your PSC and vCenter Server via the CLI can be found here.
Automating your vSphere Upgrade does not have to be scary or hard, as we went through the steps its actually quite easy. Now that we have our PSC and vCenter Server updated we can now proceed to our next step which is automating the upgrade of our ESXi hosts.
7 Blog: Upgrading vCenter Server via the GUI Installer
vSphere Upgrade Series Part 2: Upgrading vCenter
In part 1 of the vSphere Upgrade Series, Preparing to Upgrade, we covered getting started with our prerequisites, compatibility, and also prepared the vSphere Update Manager (VUM) server to migrate its data to the VCSA 6.7 during the upgrade. In part 2 we will cover the vCenter Server Upgrade to 6.7. Let’s begin.
vCenter Server Upgrade
Now that VUM has passed its Migration Assistant pre-checks, we can move to the vCenter Server Upgrade. I am also assuming here that you have a backup of the vCenter Server prior to upgrading.
We begin by mounting the VCSA installation ISO to an Admin workstation that is on a routable network to the vCenter Server we will be upgrading. Browse the ISO and open the “vcsa-ui-installer” folder then the corresponding folder for your OS. I am running this from a
Windows system so I will open the “win32” folder.
Next run the application “installer.exe” with Administrator Rights. This will launch the vCenter Server Appliance Installer to begin Stage 1.
NOTE: The upgrade of the vCenter Server is broken up into two stages, Stage 1 & Stage 2. Stage 1 is the deployment of a new VCSA and Stage 2 is where all of the configuration data and inventory are imported into the newly upgraded vCenter Server.
Begin by clicking Upgrade.
Step 1: This is the introduction and an explanation of the two Stages of the Upgrade. Click Next to continue.
Step 2: Review the EULA, check the box to accept the terms of the license agreement, and then click Next to continue.
Step 3a: Enter the source vCenter Server that you will be Upgrading by its FQDN or IP address. Click Connect To Source to reveal the additional fields.
Step 3b: Complete each required field for SSO as well as the information about the ESXi host that manages the source vCenter Server. Then click Next to continue.
Step 3c: You will prompted to verify the SSL Certificates. Review and click Yes to continue.
Step 4a: Specify the target host or vCenter Server to which the new VCSA will be deployed to. Click Next to continue.
Step 4b: When prompted, review the SSL Certificate and thumbprints then click Yes to continue.
Step 5: Specify the Virtual Machine name (this is only the Inventory name) and set the password for the VCSA that will be deployed during the upgrade. Click Next to continue.
NOTE: During an upgrade of the source VCSA, a new VCSA virtual machine is deployed and configurations are imported to the new vCenter Server Appliance. The source VCSA will be powered off and should be either removed from inventory, or have its network adapter disabled after the upgrade completes.
Step 6: Select the deployment size for the vCenter Server. If more storage is needed than the default sizing, choose the “Storage Size” dropdown for more choices. Storage size changes will be reflected in the table below the selection dropdown in the Storage (GB) column.
It may be necessary to edit the storage size from Default to Large or XLarge if importing the optional Historical & Performance Data (see image for more details).
NOTE: If you plan to use vCenter High Availability (VCHA) after you upgrade your vCenter Server, the smallest deployment size supported for VCHA is Small.
Click Next to continue.
Step 7: Select the datastore location for the vCenter Server Appliance.
The option to also Enable Thin Disk Mode is available, if you require it.
Click Next to continue.
Step 8: Configure the temporary network settings that are required to deploy the new VCSA. Be sure to add at least 1 DNS server. Once all required fields are completed, click Next to continue.
Step 9: Review all settings of Stage 1 prior to Upgrade. Once verified, click Finish to continue and kick off Stage 1 of the vCenter Server
Stage 1 of deploying the new vCenter Server Appliance is now underway.
Once the VCSA is deployed and all RPM installs are completed in
Stage 1, you can click Continue to move on to Stage 2.
Stage 2 of the upgrade is when the data from the source vCenter Server as well as vSphere Update Manager (VUM) is imported into the newly deployed VCSA.
Step 1: Review the details of the Stage 2 process and then Click Next to continue.
This will kick off a series of pre-checks on the source vCenter Server.
Step 2: Once pre-checks have completed, results will be shown on screen. Review any warnings given, as well as the resolutions to these warnings. In my upgrade scenario I had a few warnings, one reminding me to change DRS which we did before we began, and others that I validated were ok to proceed in this situation.
Review and click Close to continue.
Step 3: Select the data that you will require to be imported. The
Inventory & Configuration data is moved by default, any historical data (events, tasks, performance, etc) is optional to import. This is offered to shorten the upgrade and migration of data into the new VCSA.
Make the required choice, and then click Next to continue.
Step 4: Join the VMware Customer Experience Improvement Program or CEIP. Joining this program is optional but when you do join, it helps VMware to improve our products and services, fix problems and advise you on how best to deploy and use our products. CEIP is also required to enable the vSAN health check services.
Once a choice has been made, click Next to continue.
Step 5: Review all setting choices here and once complete, click Finish to continue.
The prompt here is a reminder that the source vCenter Server will be powered down once all network configuration is enabled on the destination VCSA.
Click OK to continue.
Stage 2 begins. Data is exported from the source vCenter Server and prepared for import.
Next, vCenter Server services are started on the destination VCSA.
Last, the copied data from the source vCenter Server is imported to the destination VCSA.
When all data has been imported to the destination VCSA, the process is complete. Messages are presented at this step as informational, such as a notice about TLS changes in vSphere 6.7.
Review these notices then click Close to continue.
Stage 2 of the vCenter Server Upgrade is now completed.
Click Close to continue.
Closing the installer will launch the vCenter Server splash page allowing you to login via the vSphere Client on HTML5.
Since the vSphere Web Client (Flex) is now deprecated, I will use the HTML5 vSphere Client because it is now our default client and it contains 95% of all workflows as compared to vSphere 6.5 Update 1 that contained 90% of workflows. The HTML5 vSphere Client will be fully featured by the Fall of 2018.
Click the button “Launch vSphere Client (HTML5)” to continue.
Enter the administrator credentials (SSO administrator or other administrator with access to vSphere) and login to view the vCenter Server and hosts.
Now we will verify the vCenter Server is now running on version 6.7. We do this by clicking on the inventory name of the vCenter Server and viewing the Version Information from the Summary tab.
Since we disabled DRS during our preparing to upgrade post, it should be enabled once again. We begin this task by highlighting the cluster and then clicking on the Configure tab to view vSphere DRS settings.
If you had set DRS to Manual versus disabling, now is the time to change that also.
Next click Edit to continue.
In the Edit Cluster Settings window, click the slider switch to enable vSphere DRS then click OK to save. DRS is now enabled on the cluster.
NOTE: You can also quickly perform these actions via PowerCLI with the ‘Set-Cluster’ cmdlet.
Set DRS automation level to Fully Automated:
After the vCenter Server Upgrade is completed, it is important to call out that power off operations only happen automatically for the vCenter Server, the VUM server must be powered off manually.
As a best practice, I suggest removing the Network Card from the old vCenter Server as well as renaming the VM to differentiate it within inventory and mitigate any accidental power on operations. Notice the new VCSA (VCSA67) virtual machine inventory name that was given. Note that renaming the VM does not change the FQDN of the VCSA, this is just the inventory name of the VM.
Lastly, be sure to power off the VUM server since it is no longer required to be running.
Did your maintenance window close or maybe you encountered another issue during an Upgrade? Not to worry, rollback is quite simple. In this vSphere environment that we just upgraded we have no external PSCs, only the vCenter Server Appliance to worry about. If we did have an external PSC, we would first power off the newly deployed
PSC, restore the PSC instance from backup, and if it was joined to an Active Directory domain, re-join it to the domain.
In our case without an external PSC we would, power off the newly deployed vCenter Server Appliance 6.7, bring the old vCenter Server Appliance 6.0 instance online (If already powered off, simply power it on. If not powered off, a restart is required), if it was joined to an Active Directory domain, it may need to be joined again (if your vCenter Server was Windows, be sure to have a local account on the server and do not rely on any cached credentials). Lastly we wait for all vCenter Server services to start and log in to the vSphere Web Client to verify the vSphere inventory.
Please review KB2146453 for more guidance on rollbacks.
Remember; Focus, Plan, Execute!
We have successfully executed our vCenter Server Upgrade! We started by reviewing our prerequisites & compatibility, gathering our data, and then upgrading our vCenter Server and vSphere Update Manager (VUM) from vSphere 6.0 Update3 to 6.7. Our VUM was migrated to the 6.7 VCSA also during this upgrade scenario. In the next vSphere Upgrade Series post, we will move to upgrading our vSphere 6.0 ESXi hosts to vSphere 6.7 by utilizing vSphere Update Manager to remediate.
Please do not hesitate to post questions in the comments section of this blog or reach out to me directly via Twitter @vCenterNerd.