VMware vSphere ESXi 7.0 Update 1 is here. If you have Intel NUCs in your homelab you should always be very careful when updating to new ESXi releases as there might be issues. Please always keep in mind that this is not an officially supported platform.
Typically, you see problems with new major releases (eg. the Realtek problem in ESXi 7.0) but this time we seem to run into a problem with 8th Gen NUCs in the 7.0 U1 release. The Intel I219-V (6) network adapter fails to load after upgrading to ESXi 7.0 U1. When you try to do a fresh installation, it fails with the well known "No Network Adapters" error.
To be on the safe side, I'm doing a quick checkup on which NUCs are safe to update and where you have to implement a workaround.
In the meantime: Stay Calm, you can run ESXi 7.0 U1 on the 8th Gen NUC!
Good news first. ESXi 7.0 U1 comes with a driver that has out-of-the-box support for the 10th Gen NUC. That means that you no longer have to create a custom image with the Community driver to run ESXi 7.0 on the 10th Gen NUC. The bad news is that the Intel I219-V (6) network adapter, used in the 8th Gen NUC (Only the Bean Canyon is affected), does not work in ESXi 7.0 U1. All other NUCs are not affected.
For reference, these are the two drivers we are talking about:
ne1000 0.8.4-10vmw.700.1.0.15843807 <- ESXi 7.0 GA
ne1000 0.8.4-11vmw.701.0.0.16850804 <- ESXi 7.0 Update 1
Upgrade Consideration
Before you start the upgrade, check that the following issues do not affect your environment:
The USB Nic Fling does not work with 7.0U1 yet. Remove it and reboot your ESXi Host, prior to installing the upgrade.
Update 2020-10-26: USB NIC Fling v1.7 has added support for ESXi 7.0U1. Make sure to get the latest version: USB Network Native Driver for ESXi- If you plan to add an "ESXi on ARM" (eg. Raspberry Pi) to the vCenter 7.0 U1, check this workaround.
- If you are using vSphere with Kubernetes, be aware that upgrading to vCenter 7.0 U1 will trigger a 60-day Evaluation for the Namespace Management (similar to vSAN Cluster Licenses).
ESXi 7.0 Update 1 on 10th Gen (Frost Canyon) NUC
- NUC10i7FNH/NUC10i7FNK (Intel Core i7-10710U - 6 Core, up to 4.7 GHz)
- NUC10i5FNH/NUC10i5FNK (Intel Core i5-10210U - 4 Core, up to 4.2 GHz)
- NUC10i3FNH/NUC10i3FNK (Intel Core i3-10110U - 2 Core, up to 4.1 GHz)
If you want to do a fresh install of ESXi 7.0 U1 on the 10th Gen NUC, just use the stock image provided by VMware. The community driver is no longer required.
If you want to upgrade from ESXi 7.0 with the latest community package, follow the following steps:
- Install the Patch and reboot
# esxcli network firewall set --enabled false # esxcli software profile update -p ESXi-7.0.1-16850804-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml # esxcli network firewall set --enabled true
- Reboot the system
- Remove the old VIB
esxcli software vib remove -n ne1000-intelnuc
- Done
Note: If you are using the community driver with the old name (not the latest, which has been renamed to ne1000-intel) you can skip Step 2. Just perform the upgrade and reboot.
ESXi 7.0 Update 1 on 8th Gen (Bean Canyon) NUC
- NUC8i7BEH/NUC8i7BEK (Intel Core i7-8559U - 4 Core, up to 4.5 GHz)
- NUC8i5BEH/NUC8i5BEK (Intel Core i5-8259U - 4 Core, up to 3.8 GHz)
- NUC8i3BEH/NUC8i3BEK (Intel Core i3-8109U - 2 Core, up to 3.6 GHz)
New Installation
If you want to do a fresh install of ESXi 7.0 U1 on the Bean Canyon, you have to create a custom image, including the old driver 0.8.4-10vmw.700.1.0.15843807.
When using PowerShell, use the following commands to remove the ne1000 from the Image Profil, and add the old version. To do so, just append the version when adding the Software Package to the Image Profile.
Remove-EsxSoftwarePackage -ImageProfile $newProfile -SoftwarePackage "ne1000" Add-EsxSoftwarePackage -ImageProfile $newProfile -SoftwarePackage "ne1000 0.8.4-10vmw.700.1.0.15843807"
Here is a working example that creates an ESXi 7.0 U1 Image with the required ne1000 driver:
$baseProfile = "ESXi-7.0.1-16850804-standard" # See https://www.virten.net/vmware/vmware-esxi-image-profiles/ for available Image Profiles Add-EsxSoftwareDepot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml $newProfile = New-EsxImageProfile -CloneProfile $baseProfile -name $($baseProfile.Replace("standard", "nuc")) -Vendor "virten.net" Remove-EsxSoftwarePackage -ImageProfile $newProfile -SoftwarePackage "ne1000" Add-EsxSoftwarePackage -ImageProfile $newProfile -SoftwarePackage "ne1000 0.8.4-10vmw.700.1.0.15843807" Export-ESXImageProfile -ImageProfile $newProfile -ExportToBundle -filepath "$($baseProfile.Replace("standard", "nuc")).zip" Export-ESXImageProfile -ImageProfile $newProfile -ExportToIso -filepath "$($baseProfile.Replace("standard", "nuc")).iso"
Note: If you get the "WinError 10054" message when exporting as ISO, just remove the online depot and try again. It should work, as the image is still cached:
Remove-EsxSoftwareDepot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml Export-ESXImageProfile -ImageProfile $newProfile -ExportToIso -filepath "$($baseProfile.Replace("standard", "nuc")).iso"
You can use the ESXi-7.0.1-16850804-nuc.iso image to install ESXi 7.0 U1 on your 8th Gen NUC. You can also use the .zip bundle to upgrade using esxcli.
Upgrade from ESXi 7.0
If you already have ESXi 7.0 installed, the quickest way to upgrade is by using VMwares online repository. Alternatively, if your ESXi has no Internet access, for example, you can also use the .zip bundle mentioned above.
Follow the following steps to upgrade ESXi 7.0 to 7.0U1 on your Bean Canyon NUC:
- Put the host into maintenance mode
- Install the patch, but do not reboot yet
# esxcli network firewall set --enabled false # esxcli software profile update -p ESXi-7.0.1-16850804-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml # esxcli network firewall set --enabled true
- Install the old ne1000 driver
# esxcli software vib install -n ne1000:0.8.4-10vmw.700.1.0.15843807 -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
- Reboot
- Your system should now reboot and come up with ESXi 7.0 U1. If any issues happen during the reboot, ESXi will automatically perform a rollback. Make sure to check the version after the reboot.
ESXi 7.0 Update 1 on any other Intel NUC
Any other NUC can be upgraded without issues.
- Install the Patch and reboot
# esxcli network firewall set --enabled false # esxcli software profile update -p ESXi-7.0.1-16850804-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml # esxcli network firewall set --enabled true
- Done
Greetings, when I upgrade my 4 Quad Network Adapter so working as it was not on the HCL. It sucks but I was able to use onboard NIC to keep environment running. I have HP DL360 G8 Servers running in my lab.
Thanks for this, updated on my 10th gen NUC!
Hi, thanks for this guide. I'd like to install ESXi 7.0 Update 1 on 8th Gen (Bean Canyon) NUC with a NVMe SSD. How can I add both ne1000 0.8.4-10vmw.700.1.0.15843807 and nvme 1.2.1.34-1vmw.650.2.50.8294253 ?
Thank you! I just tried to update my NUC8i5BEH from 7.0b to 7.0U1 and hit this problem. I then panicked! Thankfully I fond this article.
THANK YOU!!!!!!
Do you know if this is a long term fix? e.g. will the ne1000 drive problem be fixed?
Fresh install failed for me on NUC10i7FNH. Installer is unable to format the vmfs6 partition and installation crashes. Does anyone got this problem ? I tried with a patched 7.0.0b (network adapter) and it installation worked like a charm.
Any idea ?
Hi,
The problem is solved, my SDD was in frozen state. Now, the ESXi 7.0 U1 is running perfectly !
I have this exact issue now, what you mean by "SSD was in frozen state"? How did you resolve this?
I will explain you my issue which finally is not totally solved.
My config:
BXNUC10i7FNH2 with 1 TB INTEL SSDPEKNW01 (NVMe M.2)
and Samsung 860 PRO 1TB (SATA).
I installed ESXi 7.0 on my Samsung disk and the Intel NVMe disk was a spare one. But sometimes my VM was freezing and I decided to migrate to 7.0. U1 with a fresh install.
The problems were beginning with a i/o error at 5 % progress when I tried to install it. Then I decided to delete the partition table and to create a new one without any result. Same error at 5%. With a linux rescue disk utility I discovered my SSD was frozen. I decided to perform a secure erase with the Samsung bootable USB utility. It worked and I was able to install ESXi again without any error. Everything seemed ok until I tried to migrate my VM to the new created datastore. I was unable to copy anything inside with still an i/o error !! Finally I succeeded to copy my VM with many difficulties but I resigned to make work my VM on an instable disk because of too much i/o and snapshots issues.
2 possible issues:
The NUC SATA controller is out.
VMW-AHCI driver fails with Samsung disks.
I will try this weekend to install a 7.0 version to see if error remains.
I also made ESXi 7.0 U1 install on the Intel NVMe disk (no problem at all) and then tried to create a new datastore on the Samsung disk. Sometimes it worked and sometimes it failed. And copy any date on this disk was a pain with plenty of i/o errors. I also tried with an other new disk. So the i/o errors does not come from the disk.
Good luck for all and if you have some solutions, there are welcome ;-))
My installation issue is exactly same and also with Samsung SSD, in this case PM863a drive. These drives work fine with 7th Gen NUC so it must be something AHCI interface related..
Since the 7.0.0b version, I see that it is a pain to format and mount a VMFS6 partition (datastore) properly on Samsung SSD devices.
I also have a « clone » device with exactly the same configuration which works properly without any I/o error.
The main difference is that the Samsung datastore2 (single VMFS6 partition) was created under ESXi 7.0.0 working on Intel NVMe disk.
Since, I made a fresh 7.0 U1 install on the Intel disk but the Samsung datastore was untouched.
I will try to make a VMFS6 then a datastore on my Samsung SSD under v7.0.0 and put it back to the NUC to see his behavior.
Hey,
similar problem.
But even if you manage to create datastore and mount it. It is very unstable. You create a VM and started it, but if you try to copy a file from NVMe to this drive it fails with input/output error.
Drive was checked in Windows OS and works perfectly (I installed Windows on NUC and checked the drive).
Replying to this message as I cannot reply to other thread. Yes SATA cable is seriously flawed and SATA is completely unusable. We did a RMA for all our 10th Gen NUCs and switched them to 8th Gen models, no issues at all with these.
Hi,
Check the SATA cable connection. Unplug and plug it again firmly. It should solve the problem as my NUC worked well again since I did it. Me too I couldn't believe it was the cause of the issue.
Please also, send a message to the Intel support to make them know their cable system is so bad.
Thanks for the answers, Ced and tphakala.
The reason why I doubt this is a cable fault is that I installed Windows 10 on SATA SSD without a problem. I used heavy simultanous copy operations (local and network), and they worked without a problem.
The issue happens only with ESXi. If that was a cable, Windows would have problems as well. Also, the problems would be more consistent.
I have managed to do the following on ESXi and SATA SSD:
- upload big ISO over web UI
- create 2 VM and run them simultaniously
- perform multiple VM clone operations on this drive too.
However, the following always fails with input/output errors:
- copy big files (over 5GB .ISO) over SSH
- copy local big .ISO files from NVMe to SATA SSD using "cp" command
- restoring VEEAM backup to SATA SSD.
Therefore, I would recommend check kernel log (on web UI or output of DMESG command) to check whether you still do not get input/output error messages. Also, whether above failing operations fail for you also.
For me, this is a clear ESXi AHCI driver problem.
thanks
Rafal
Downgraded to 7.0.0 VMW-AHCI driver. same issue :-((
I will try again after a new secure erase.
Well it was indeed a material error! When you have to disassemble the hood of the NUC to insert the SSD, do not pull too much on the cable connected to the motherboard!
Unless Intel mis-assembles the cable from the start. There is fake contact and you get lots of i/o errors (normal issue !!)! I disconnected the cable completely and then reconnected correctly. Well now everything works wonderfully!!
Oh wow, I need to check this! Thank you so much for debugging.
Let me know my friend ! ;-))
I took NUC apart and even though there was nothing visibly wrong with SATA connection it was the issue in my case also. I had to secure SATA cable twice before installation finished correctly.
SATA cable on these 10th Gen NUCs seems to be extremely flaky, SATA drives should be avoided with this model :-(
Are you sure that was the cable problem?
After several attempts, a datastore will be created and you can use in limited way. However, if you try to move big files (4GB) over SSH, or copy such a big file from another disk (for example: NVMe), it should fail with input/output error.
Check DMESG output on ESXi. It should have plenty input/output errors :(
I replaced my Samsung SSD with Crucial MX500.
No more I/O errors. Everything works smoothly.
Good to hear that from you ! Once the cable is firmly fixed there will be no more problem. Now, I will not disassemble the NUC anymore !! I will tell to the Intel support. This issue IS NOT admissible.
Rafal R could you please post the exact Crucial MX500 model you used.
Hi,
Can you please post an update with instructions for the ESXi 7.0U1b on the NUC8?
Thanks!
I would like this also please :)
Hi! love your blog btw, I have been trying to upgrade my 6.5 ESXi Intel Nuc for some time as I am repeatedly experiencing the ssd being dropped.. Now when running the command you mention above to upgrade to 7.0 for any other NUC I get the following response and have not been able to figure out why "Got no data from process: LANG=en_US.UTF-8 /usr/lib/vmware/esxcli-software profile.update -d "https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml" -p "ESXi-7.0.1-16850804-standard"
"
Do you know why I cant upgrade?
That can happen when there is no space to download the package or the firewall is blocking connections.
Is the firewall configured to allow HTTP traffic?
# esxcli network firewall ruleset set -e true -r httpClient
Do you have configured a persistent scratch location? If no, see:
https://kb.vmware.com/s/article/1033696
There should be enough space as there is 10GB free on the usb memory stick that esxi is installed on. I have tried it with both the "# esxcli network firewall set --enabled false" as well as the specific command you suggested.
How do I tell if I have a persistant scratch location? In the KB it says that vmware creates this by default. If I want to create it now I need to create it on a iSCSI drive that I have, is that detrimental to the performance?
There should be enough space as there is 10GB free on the usb memory stick that esxi is installed on. I have tried it with both the "# esxcli network firewall set --enabled false" as well as the specific command you suggested.
How do I tell if I have a persistant scratch location? In the KB it says that vmware creates this by default. If I want to create it now I need to create it on a iSCSI drive that I have, is that detrimental to the performance?
There are multiple partitons on the USB boot drive, so there is no "10GB free" in general.
Run
This is the response. Matches with the change I did. I moved the existing scratch disk to a iSCSI datastore that I have on my NAS:
(vim.option.OptionValue) [
(vim.option.OptionValue) {
key = "ScratchConfig.ConfiguredScratchLocation",
value = "/vmfs/volumes/495d040b-8df271b6-934d-f44d306ef49b/.locker-esxi"
}
]
The update process gave the same error message on both scratch disks.
Just to be sure - You rebooted the host after setting the scratch location?
Are there any errors in /var/log/esxupdate.log?
If that doesn't help, you could download the 7.0 upgrade bundle and install it from a local copy, instead of downloading it.
I ran into this issue and had to run
esxcli software sources vib list --depot=https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | grep tools-light | sort
I found it on
https://tinkertry.com/easy-update-to-latest-esxi#apr-14-2019-update
Thankyou ! up and going on my NUC8i7BEH after first attempt failed
Anyone getting this error exporting the iso?
Export-ESXImageProfile : The read operation timed out
yep - same here
I had problems with your update command I had to use
esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-7.0.1-16850804-standard
Does anyone know if EXSI 7.0 Update 1c will work on the Gen boards? I have the NUC8i5BEH and managed to get networking to work following this lifesafer page. But am I now stuck on 7.0 or can I upgrade to 7.0 Update 1c?
Many thanks.
There is no new ne1000 driver in 7.0 U1c, so the 8th Gen NUC network adapter problem has not changed. Updating will again overwrite the ne1000 driver, but you can use the same method to reinstall the working driver.
- Update to 7.0u1
- Install ne1000:0.8.4-10vmw.700.1.0.15843807
- Reboot
Thank you Florian
Thank you. Update 1c installed successfully with these two commands:
esxcli software profile update -p ESXi-7.0U1c-17325551-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
esxcli software vib install -n ne1000:0.8.4-10vmw.700.1.0.15843807 -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
I have nuc10i7FNH3. Samsung 860 EVO 250Gb. ESXi 7.0 GA
I have inconsistent behavior. I am unable to create Datastore. Once copying over nvme_pci.v00 from ESXI 6.5 allowed me to create datastore, but i was unable to run any VMs
I see Rafal R had similar behavior and got fixed after moving to Crucial MX500. I will try the same and post my results. Thanks for great discussions
"Here is a working example that creates an ESXi 7.0 U1 Image with the required ne1000 driver:"
does not work for me, several tries.
"Note: If you get the "WinError 10054" message when exporting as ISO, just remove the online depot and try again. It should work, as the image is still cached:"
does not work for me, several tries.
Use only V7.0.0. 7.0b or U1 won't work.
I downloaded the offline bundle from vmware. Using pkg and izip for the customizer.
Which NUC are you talking about?
Some NUCs need special Network adapters to work with 7.0u1.
Hi,
Back after 3 months of intensive use of my NUC with datastores created on my Samsung SSD. Since the cable issue, all is still working perfectly without any error. Nevertheless a mystery is remaining regarding some of comments. Does somebody can explain it now ?
Can anyone chekc if the ESXi 7.0 update 2 patch still has the same problem? Seems like there is a new ne1000 driver...