In case you want to migrate running or suspended virtual machines from one ESXi host to another in vSphere, by using live migrations or so called vSphere vMotion, the target hosts processor should support the same CPU instructions to the virtual machine after migration that the processor of the source host provided before migration.

When you have ESXi hosts in your vSphere cluster, they will use different underlying CPU architectures, you can enable Enhanced vMotion Compatibility (EVC) on the cluster to support vMotion (live migrations) despite of the different CPU architectures.

Note that the EVC Mode nevertheless requires that all ESXi Hosts needs to use the same CPU vendor, either all AMD or Intel.


For this post I will use in my lab environment for example two IBM x3650 server. The first one is an IBM x3650 M4 and using an Intel Xeon(R) CPU E5-2650 @ 2.00Ghz (2-socket processor) and the second is an IBM x3650 M4 and using an Intel Xeon(R) CPU E5-2630 v3 @ 2.40GHz (2-socket processor).

So they will not having the same exact CPU architecture and feature set and will therefore not support vMotion (live migrations) by default without enabling the EVC mode.

Configure EVC to ensure that virtual machine migrations between hosts in the cluster do not fail because of CPU feature incompatibilities.

Source: https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vcenterhost.doc/GUID-6698B0EE-8B52-462B-88FF-ED90666D88E2.html


Table Of Contents
  1. Introduction
  2. Enable EVC Mode
  3. Disable EVC Mode
  4. Links


Introduction

vCenter Server performs compatibility checks before it allows migration of running or suspended virtual machines to ensure that the virtual machine is compatible with the target host.

vMotion transfers the running state of a virtual machine between underlying ESXi systems. Live migration requires that the processors of the target host provide the same instructions to the virtual machine after migration that the processors of the source host provided before migration. Clock speed, cache size, and number of cores can differ between source and target processors. However, the processors must come from the same vendor class (AMD or Intel) to be vMotion compatible.

Migrations of suspended virtual machines also require that the virtual machine be able to resume execution on the target host using equivalent instructions.

When you initiate a migration with vMotion or a migration of a suspended virtual machine, the Migrate Virtual Machine wizard checks the destination host for compatibility. If compatibility problems prevent migration, the wizard displays an error message.

The CPU instruction set available to the operating system and to applications running in a virtual machine is determined at the time that a virtual machine is powered on. This CPU feature set is based on the following items:

  • Host CPU family and model
  • Settings in the BIOS that might deactivate CPU features
  • ESXi version running on the host
  • The compatibility setting of the virtual machine
  • The guest operating system of the virtual machine


To improve CPU compatibility between hosts of varying CPU feature sets, some host CPU features can be hidden from the virtual machine by placing the host in an Enhanced vMotion Compatibility (EVC) cluster. 

For more information about EVC, see About Enhanced vMotion Compatibility.

Source: https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vcenterhost.doc/GUID-03E7E5F9-06D9-463F-A64F-D4EC20DAF22E.html





Enable EVC Mode

We can enable the EVC mode within the cluster settings and here under the Configure tab.

Cluster -> Settings -> Configure -> Configuration -> VMware EVC


I will need to enable EVC mode for  Intel® Hosts.


To enable this feature we must first shutdown and power-off all virtual machines.

The host cannot be admitted to the cluster’s current Enhanced vMotion Compatibility mode. Powered-on or suspended virtual machines on the host may be using CPU features hidden by that mode.


In my case to enable this feature I will first need to remove both ESXi Hosts from the cluster because I can’t just power-off the vCenter Server Appliance which I need in running state to enable the EVC mode.


Now as both ESXi hosts are not anymore member of the cluster, I can enable the EVC Mode. I will now select the highest EVC level of the ESXi host which is using the oldest CPU (my IBM x3650 M4) for the CPU Mode which is in my case the Intel Sandy Bridge Generation. You can use the following article to find the applicable EVC mode.

Intel CPU EVC Matrix (VMware Enhanced vMotion Compatibility)
https://www.virten.net/2013/04/intel-cpu-evc-matrix/



I will now move both ESXi hosts back to the cluster. From now on vMotion should work.

Below I will try to migrate a virtual machine which is running on the esxi-02.braincourt.de host to the esxi-01.braincourt.de host.








The virtual machine was migrated successfully to the esxi-02.braincourt.de host by using vSphere vMotion.



Disable EVC Mode

Disabling the EVC mode will work despite virtual machines are running in the cluster. Just for re-enable EVC it might be necessary to power off running virtual machines.



I will not try to migrate the virtual machine we previously migrated to the esxi-01 host back to the esxi-02 host by using a live migration with vMotion.

Because both ESXi hosts have different Intel CPUs, the live migration by using vMotion will fail. Actually directly after disabling the EVC mode in vCenter, I was still able to use live migrations till I rebooted both hosts.

After rebooting both ESXi hosts the following error message below appears when trying to use live migrations what is expected.

The target host does not support the virtual machine’s current hardware requirements.
Use a cluster with Enhanced vMotion Compatibility (EVC) enabled to create a uniform set of CPU features across the cluster, or use per-VM EVC for a consistent set of CPU features for a virtual machine and allow the virtual machine to be moved to a host capable of supporting that set of CPU features.





Links

CPU Compatibility and EVC
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vcenterhost.doc/GUID-03E7E5F9-06D9-463F-A64F-D4EC20DAF22E.html

Change the EVC Mode for a Cluster
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vcenterhost.doc/GUID-6698B0EE-8B52-462B-88FF-ED90666D88E2.html

Intel CPU EVC Matrix (VMware Enhanced vMotion Compatibility)
https://www.virten.net/2013/04/intel-cpu-evc-matrix/

VMware EVC Mode – A Complete Overview
https://www.bdrsuite.com/blog/vmware-evc-mode-overview/