March 09, 2021

Faster vMotion Makes Balancing Workloads Invisible

vSphere 7 Update 2 automatically tunes vMotion to fully utilize the available network bandwidth for faster live-migrations. There's no more manual tuning required like with previous vSphere versions.

New in vSphere 7 Update 2 is that vMotion is able to fully benefit from high-speed bandwidth NICs for even faster live-migrations. Manual tuning is no longer required as it was in previous vSphere versions, to achieve the same.

vMotion History

vMotion Stream Architecture

vSphere vMotion, in versions prior to vSphere 7 Update 2, does not saturate high bandwidth NICs out-of-the-box like 25, 40, and 100 GbE. It required tuning at multiple levels such as vMotion streams, the TCP/IP VMkernel interfaces, and potentially even the NIC driver. That’s because, by default, vMotion would use a single stream for handling the vMotion process. Click here to learn more about the vMotion Stream Architecture.

One stream consists of a competition helper, crypto helper, and stream helper. One helper equals one CPU core. Typically, one stream helper is capable of saturating ±15 GbE of bandwidth. This is great for 10 GbE NICs, but it becomes a limiting factor when using 25 GbE or higher NICs. With 25, 40 and even 100 GbE becoming mainstream quickly, vMotion needs to be able to utilize these bandwidths automatically for more efficient live-migration operations.

vMotion Multi Streams

In previous vSphere versions, we provided ways to instantiate more streams to utilize more CPU power to use more network bandwidth for vMotion. Mainly by configuring advanced settings. That resulted in faster live-migrations as more bandwidth is provided to vMotion to replicate memory pages.

Note: make sure that if you used the manual advanced settings before, that you revert to the default settings to ensure you'll benefit from the Auto Stream capability in vSphere 7 Update 2.

Automatically Adjust for High Bandwidth

With vSphere 7 Update 2, the vMotion process automatically spins up the number of streams according to the bandwidth of the physical NICs used for the vMotion network(s). This is now an out-of-the-box setting. The usable bandwidth for vMotion is determined by quering the underlying NICs.

This means that all VMkernel interfaces enabled for vMotion are checked, and the underlying physical NIC bandwidths. Depending on the outcome, a number of streams is instantiated. The baseline is 1 vMotion stream per 15 GbE of bandwidth. This results in the following number of streams per VMkernel interface;

  • 25 GbE = 2 vMotion streams
  • 40 GbE = 3 vMotion streams
  • 100 GbE = 4* vMotion streams

*4 is the max nr of vMotion streams that make sense because of a limit of 4 RX queues

To Conclude

vSphere vMotion keeps on getting better and better. The beauty of the updates is that you'll benefit from the improvements automatically by installing, or upgrading to, vSphere 7 Update 2. Be sure to check out this page for more details about vMotion, including all the updates since vSphere 7.

Associated Content

home-carousel-icon From the action bar MORE button.

Filter Tags

Application Acceleration vSphere vSphere 7 Storage vMotion vMotion Blog Intermediate Migrate