VMware EVC Mode to Enable Intel Gen5-Gen10 NUC vMotion

Many VMware Homelabs are based on Intel NUCs. It is also very common that generations are mixed which can lead to compatibility issues when trying to vMotion VMs across different generations. This is typically where VMware EVC comes into play.

VMware EVC creates a baseline of CPU instructions for virtual machines running on ESXi hosts. When you add newer Hosts, EVC hides the new CPU instructions to the virtual machines. While this works great for Xeon CPUs used in professional servers, it has some limitations with consumer CPUs used in the Intel NUC ecosystem.

The problem has become worse with the latest 10th Gen Comet Lake/Frost Canyon NUC. Despite having a 10th generation CPU, it requires the EVC baseline to be configured to "Sandy Bridge", which is the 2nd generation of Intel Core-i CPUs:

  • NUC10i7FNH/NUC10i7FNK (Intel Core i7-10710U - 6 Core, up to 4.7 GHz)
  • NUC10i5FNH/NUC10i5FNK (Intel Core i5-10210U - 4 Core, up to 4.2 GHz)
  • NUC10i3FNH/NUC10i3FNK (Intel Core i3-10110U - 2 Core, up to 4.1 GHz)

When you try to activate VMware EVC higher than Sandy Bridge, the following error message is displayed:

Compatibility
The host's CPU hardware does not support the cluster's current Enhanced vMotion Compatibility mode. The host CPU lacks features required by that mode.

When you try to add the Host to an EVC Enabled Cluster, the task fails:

Operation failed!
The host's CPU hardware does not support the cluster's current Enhanced vMotion Compatibility mode. The host CPU lacks features required by that mode.
CPUID faulting is not supported.
See KB 1003212 for more information.
Host is of type: vendor intel family 0x6 model 0xa6

A common EVC Mode that allows Gen5 - Gen8 NUCs to be used in the same Cluster is "Haswell". To add a 10th Gen NUC, you have to lower EVC to Sandy Bridge. The problem with the "Frost Canyon" is that it does not support the CPUID faulting feature, required since EVC "Ivy Bridge".

Generation 6-8 NUCs are very generous when it comes to vMotion compatibility. They have a similar feature set and you can migrate VMs without EVC. My first Idea was to retire the 5th Gen NUC and disable EVC. This could be a solution because the CPUID faulting feature can be disabled in the Virtual Machines configuration by adding the following line to the vmx configuration file:

cpuidFaulting.enable = "FALSE"

With "CPUID faulting" disabled, I can migrate VMs from Gen6-8 to Gen10, but unfortunately not the other way as the Gen10 has 4 new features:

  • MDS_NO
  • RSBA_NO
  • IBRS_NO
  • RDCL_NO

As a result, I tried to create my own EVC Mode to allow Gen5 - Gen10 NUCs to be in the same Cluster. Basically, you only need to remove Restricted Transactional Memory (RTM), Hardware Lock Elision (HLE) and CPUID faulting from the Broadwell EVC Mode to be compatible with all of my NUCs.

If you have the same problem, here is how you can do the same. Please keep in mind that the file might be overwritten when you patch your vCenter Appliance, so you have to copy it again. And of course, it's fully unsupported.

  1. Download EVC XML File (Right-Click -> Save as)
    vSphere 6.7: evcModes67-NUC.xml
    vSphere 7.0: evcModes70-NUC.xml

  2. Make a backup copy of the old file:
    # cd /usr/lib/vmware-vpx/
    # cp evcModes.xml evcModes.xml.backup
  3. Copy the new XML to the vCenter Service Appliance at /usr/lib/vmware-vpx
  4. Replace the original XML file with the one downloaded for your vCenter version
    vSphere 6.7
    # cp evcModes67-NUC.xml evcModes.xml
    
    vSphere 7.0
    # cp evcModes70-NUC.xml evcModes.xml
  5. Restart the vpxd Service
    # service-control --restart vpxd
  6. To activate the EVC Mode you have to use PowerCLI. Verify that the EVC Mode is available:
    PS> $Global:DefaultVIServer.ExtensionData.Capability.SupportedEVCMode |select key

  7. Activate the EVC Mode:
    PS> Set-Cluster NUC -EVCMode "intel-nuc"
    
  8. Verify EVC Mode:
    Get-Cluster NUC | Select Name,EVCMode
    
    Name EVCMode  
    ---- -------  
    NUC  intel-nuc

Reminder: VPXD Services fails to start when Clusters are configured with EVC mode intel-nuc and the mapping is missing (after patches). Copy the file again and restart vpxd.

  1. I think the Gemini Lake/June Canyon NUCs require a similar hack. I had to go back to Westmere to permit vMotion to/from a DL380 Gen8.

  2. Thanks for all the work you've put into this. I unfortunately have ran into an error when attempting to switch the cluster to the new EVC mode after following your steps. Wonder if you could take a look please:

    "Could not obtain the result of task '/VIServer=xxxxxx.local:443/Task=Task-task-2002/'. Task name is 'ConfigureEvcMode_Task'. The following error occured: Unexpected error occured. Cannot convert type 'VirtualMachineFeatureRequirement' to 'Object'. Set-Cluster: 19/05/2020 16:38:08 Set-Cluster Update operation for cluster 'NUC' failed."

    Any hints or thoughts as to why this could be?

    I'm working with a NUC5i5MYHE & NUC10i7FNK.

  3. I have only Gen7, 8 & 10 NUCs. I created a new cluster and then assigned the custom EVC mode (Intel-nuc) in this article to it. When I try to add a Gen 10 NUC (since it has no VMs) to the cluster, I get "general runtime error occured" Can you please help?

Leave a Comment

NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This site uses Akismet to reduce spam. Learn how your comment data is processed.