ESXi 7 and 8 Installation fails with "Fatal CPU mismatch on feature"

When you try to install ESXi 7.0 or 8.0 on a system with a 12th Gen Intel CPU, the installation fails with a purple diagnostics screen:

HW feature incompatibility detected; cannot start

Fatal CPU mismatch on feature "Hyperthreads per core"
Fatal CPU mismatch on feature "Cores per package"
Fatal CPU mismatch on feature "Cores per die"

When you try to power on Virtual Machines on 13th Gen Intel CPUs, ESXi crashes with the following PSOD:

GP Exception 13 in world

This problem is caused by the new architecture of Intel CPUs which are equipped with different types of cores - Performance-cores and Efficient-cores. With vSphere 7.0 Update 2, the kernel parameter cpuUniformityHardCheckPanic has been implemented to address the issue. This article explains the manual and automated methods to properly implement the kernel option.

This article is valid for:

• ESXi 8.0 and later
• ESXi 7.0 Update 2 and later

Manual Method

The parameter needs to be set prior to installation and the first boot of ESXi.

1. When ESXi installation starts, press SHIFT+O to edit boot options.
2. Append cpuUniformityHardCheckPanic=FALSE
   
3. Press ENTER
4. Install ESXi
5. When the installation is finished, reboot the system and press SHIFT+O to edit the boot options again.
6. Append cpuUniformityHardCheckPanic=FALSE and press ENTER
7. To make the kernel option permanent, run the following command on your ESXi host:

   # esxcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE
   

8. For 13th Gen Intel CPUs (Raptor Lake), you also have to enable kernel option ignoreMsrFaults to prevent PSOD during VM startups. (Credit to William Lam for providing a solution for the PSOD Issue)

   # esxcli system settings kernel set -s ignoreMsrFaults -v TRUE

This setting allows ESXi to work with different P-Cores and E-Cores, present in 12th Gen Intel CPUs.

Automated Kickstart Method

If you prefer an automated approach, you can also use a scripted kickstart installation to fix the issue for you. In this example, I'm installing ESXi 8.0 using a USB flash drive on an Intel 12th Gen Pro NUC.

1. Create a bootable USB flash drive using the ESXi ISO (See this article for instructions).
2. On the flash drive, open the /efi/boot/boot.cfg file with an editor and add the following options to the kernelopt= line: ks=usb:/KS.CFG cpuUniformityHardCheckPanic=FALSE
   This will instruct the ESXi installer to perform a scripted installation from the /KS.CFG file.
3. As ESXi will not be able to boot without the kernel option and thus does not reach the %firstboot section, you have to manipulate the installed ESXi on the USB flash drive post-installation but before the first reboot. This happens in the %post section.  Create the /KS.CFG file with the following content (fix parameters to match your environment):

   vmaccepteula
   install --firstdisk=usb --overwritevmfs
   
   network --bootproto=static --ip=192.168.0.26 --netmask=255.255.255.0 --gateway=192.168.0.1 --hostname=esx12.virten.lab --nameserver=192.168.0.1
   rootpw VMware1!
   
   %post --interpreter=busybox
   
   /bin/mcopy -o -i /dev/disks/mpx.vmhba32\:C0\:T0\:L0\:5 ::BOOT.CFG /tmp/BOOT.cfg
   /bin/sed -i '/kernelopt/s/$/ cpuUniformityHardCheckPanic=FALSE/' /tmp/BOOT.cfg
   /bin/mcopy -o -i /dev/disks/mpx.vmhba32\:C0\:T0\:L0\:5 /tmp/BOOT.cfg ::BOOT.CFG
   /bin/reboot
   
   %firstboot --interpreter=busybox
   
   # Enable SSH
   vim-cmd hostsvc/enable_ssh
   vim-cmd hostsvc/start_ssh
   
   # Enable ESXi Shell
   vim-cmd hostsvc/enable_esx_shell
   vim-cmd hostsvc/start_esx_shell
   
   # Suppress Shell warning
   esxcli system settings advanced set -o /UserVars/SuppressShellWarning -i 1
   
   # Disable CPU Uniformity Check
   localcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE 
   
   # Ignore general protection faults as a result of rdmsr and wrmsr instructions
   localcli system settings kernel set -s ignoreMsrFaults -v TRUE
   # SSH Key
   cat > /etc/ssh/keys-root/authorized_keys <<EOF
   ssh-rsa AAAAB3N[...] yourname
   EOF
   
   # NTP
   esxcli system ntp set -s pool.ntp.org
   esxcli system ntp set -e 1
   

4. Insert the flash drive into your System and power it on. The installation should start automatically and after a couple of minutes, you should be able to access the system with SSH or UI.
5. Please note that the %firstboot section is not applied when Secure Boot is enabled, which is the default for most systems. Consider disabling Secure Boot for the full kickstart pleasure, or run these commands manually. Secure Boot does not affect the %post section, so the mcopy and sed magic will work.

Alternative Option - Disable Performance or Efficient Cores

If you do not want to change kernel options, you can also disable P or E cores to completely get rid of the non-uniform core configuration. For performance, I recommend disabling Efficient Cores. Make also sure to enable Hyperthreading.

1. Make sure to get the latest BIOS version for your NUC
2. Power On the NUC
3. Immediately press F2 to enter BIOS
4. Navigate to Power, Performance and Cooling and set External Ambient Temperature Tolerance to User Defined
5. Navigate to Power, Performance and Cooling > Performance > Processor
6. Set Active Efficient Cores to 0
7. Press F10 to Save and Exit