Tag Archives: Issue

Critical Patch for All-Flash vSAN with Deduplication (6.0 and 6.5)

VMware has released a critical patch for vSAN 6.0 and 6.5. This patch contains a fix for a highly critical vSAN issue that might cause IO read errors and in some cases severe symptoms such as inaccessible VMs, host failures, or stuck resyncs. The issue manifests itself only under highly specific operations and IO patterns.

The bug only affects All-Flash vSAN configurations with deduplication enabled.

Symptoms for this issues are the following messages in the Events tab:

  • "vSAN detected an unrecoverable medium or checksum error for component uuid on disk group uuid."
  • "vSAN detected and fixed a medium or checksum error for component uuid on disk group uuid."

The following versions are affected according to KB2151042 and KB2151081.

Name Patch Date Build
ESXi 6.5 Update 1 ESXi650-update1 2017-07-27 5969303
ESXi 6.5 Update 1 (Security only) ESXi650-update1 2017-07-27 5969300
ESXi 6.5d (vSAN 6.6 Patch) ESXi650-201704001 2017-04-18 5310538
ESXi 6.5 Express Patch 1a ESXi650-201703002 2017-03-28 5224529
ESXi 6.5 Patch 1 ESXi650-201703001 2017-03-09 5146846
ESXi 6.5 Patch 1 (Security Only) ESXi650-201703001 2017-03-09 5146843
ESXi 6.5a ESXi650-201701001 2017-02-02 4887370
ESXi 6.5 GA 2016-11-15 4564106
ESXi 6.0 Update 3a (Patch 5) ESXi600-201706001 2017-06-06 5572656
ESXi 6.0 Update 3a (Patch 5) (Security Only) ESXi600-201706001 2017-06-06 5485776
ESXi 6.0 Express Patch 7a ESXi600-201703001 2017-03-28 5224934
ESXi 6.0 Update 3 ESXi600-update3 2017-02-24 5050593
ESXi 6.0 Update 3 (Security Only) ESXi600-update3 2017-02-24 5047589
ESXi 6.0 Patch 4 ESXi600-201611001 2016-11-22 4600944
ESXi 6.0 Patch 4 (Security Only) ESXi600-201611001 2016-11-22 4558694

VMware states that all vSAN customers using All-Flash with deduplication enabled must upgrade to this patch immediately:

VCSA 6.5 Broken Filesystem - "Welcome to Emergency Mode"

Today I managed to crash the storage used in my home lab. After fixing the FreeNAS box, my vCenter Server Appliance (Version 6.5 Update 1) refused to boot and after opening the console, it welcomed me with the following error message:

Welcome to emergency mode! After logging in, type "journalctl -xb" to view system logs, "systemctl reboot" to reboot, "systemctl default" or ^D to try again to boot into default mode.
Give root password for maintenance
(or press Control-D to continue):

Typically, this problem is caused by filesystem issues. This article explains how to fix the filesystem and get the appliance back up.

Read more »

Heads Up - ESXi not working on 7th Gen (Kaby Lake) Intel NUC7

I've received reports that the ESXi 6.5 and ESXi 6.0 installer fails to load on the latest 7th Gen NUCs:

  • NUC7i3BNH
  • NUC7i3BNK
  • NUC7i5BNH
  • NUC7i5BNK

The main issue is that the I219-V NIC is not recognized, so the installer fails with the well known "No Network Adapters" error message. Today I managed to get my hands on a NUC7i3BNH to narrow down the issue. By now I've not managed to get the embedded Network Adapter to work. A workaround with a USB-based NIC is possible.

[Update 2017-02-25 - A fix is available]

Read more »

VMware NSX 6.2.3 has been pulled from distribution (DFW Issue)

Last week VMware warninghas silently removed the download for the latest version of VMware NSX. Today the Support Insider blog, which is a great resource for first-hand field experience, reported that the NSX for vSphere 6.2.3 release has been officially pulled.

VMware recommends to suspend updates until the replacement for NSX 6.2.3 is released.

USB 3.0 devices detected as USB 2 in ESXi 6.0 and 5.5

In my latest post USB Devices as VMFS Datastore in vSphere ESXi 6.0 I had a problem with USB 3.0 devices that are detected as USB 2 in ESXi. I know that USB 3.0, also known as eXtensible Host Controller Interface (xHCI), is supported in ESXi 6.0 and ESXi 5.5 Build 2143827 or later. Unfortunately all of my devices are detected as USB 2.1, despite the USB 3 hub was visible. This problem applies to both, USB devices in path-through mode, and USB devices mounted from the command line with usbarbitrator disabled. The solution was quite simple and not related to an ESXi, but to a UEFI configuration.
xhci-smart-auto

Read more »

More Information on CVE-2015-5177 (ESXi OpenSLP Remote Code Execution)

You might be aware of the 3 critical security issues that VMware has published and fixed a couple of days ago in VMSA-2015-0007. The information provided in the security advisory regarding the first issue, CVE-2015-5177 (ESXi OpenSLP Remote Code Execution), are:

VMware ESXi contains a double free flaw in OpenSLP's SLPDProcessMessage() function. Exploitation of this issue may allow an unauthenticated attacker to remotely execute code on the ESXi host.

Relevant Releases
VMware ESXi 5.5 without patch ESXi550-201509101
VMware ESXi 5.1 without patch ESXi510-201510101
VMware ESXi 5.0 without patch ESXi500-201510101

In this post I am trying to give a better understanding of the vulnerability and its consequences. Please note that the information in this post are my personal opinions. I cannot guarantee that these information are accurate. The main fact is that VMware has published a fix and you should install the patch to be on the safe side. In the real world, you might have something like a "change process" where you can't rollout the patch for hundreds of systems immediately. Or you have a single ESXi that you don't want to reboot at the moment. In this situation, this post tries to help...

Read more »

VMware: Thrilling Start to October 2015

VMware has started October 2015 with a bunch of patches, a security advisory and a major bug in their latest ESXi 5.5 release. Read more »

"Unsupported SVGA driver" during Windows 10 upgrade in VMware Workstation

windows-10-upgradeUpgrading Windows 7, 8 or 8.1 to Windows 10 in VMware Workstation fails with the following error message:

Unsupported SVGA driver

The issue only applies to the Microsoft Update Validation Tool which has problems with the VMware SVGA driver. To resolve this issue, use the Windows 10 ISO to upgrade your Virtual Machine.

Read more »

How to fix ESXi on Intel NUC malformed Manufacturer and Model

VMware ESXi 5.x and 6.0 installed on a 5th gen Intel NUC (NUC5i7RYH, NUC5i5RYH, NUC5i5MYHE, NUC5i5RYK, NUC5i3RYH, NUC5i3MYHE and NUC5i3RYK) has a problem with SMBios system information.

  • Manufacturer and Model in the vSphere Client is displayed with replacement characters (� black diamond with a white question mark).
    5th-gen-nuc-manufacturer
  • Adding host to a vCenter Server fails with the following error message

    A general system error occurred: at line number 7, not well-formed (invalid token)

To solve this problem you have to replace the BIOS. This post explains how to create a customized BIOS with changed SMBios variables and install it on your NUC.

Read more »

VMware ESXi 5.x E1000 PSOD Issue

Even though the bug has been fixed I still see it to come up. ESXi 5.x has two issues that might cause the ESXi host to fail with a purple diagnostic screen. Both PSOD look very similiar:

KB2059053:

#PF Exception 14 in world wwww:WorldName IP 0xnnnnnnnn addr 0x0
PTEs:0xnnnnnnnn;0xnnnnnnnn;0x0;
0xnnnnnnnn:[0xnnnnnnnn]E1000PollRxRing@vmkernel#nover+ 0xdb9
0xnnnnnnnn:[0xnnnnnnnn]E1000DevRx@vmkernel#nover+0x18a

KB2079094:

#PF Exception 14 in world wwww:WorldName IP 0xnnnnnnnn addr 0x0
PTEs:0xnnnnnnnn;0xnnnnnnnn;0x0;
0xnnnnnnnn:[0xnnnnnnnn]E1000PollRxRing@vmkernel#nover+0xeb7
0xnnnnnnnn:[0xnnnnnnnn]E1000DevRx@vmkernel#nover+0x18a
[...]
0xnnnnnnnn:[0xnnnnnnnn]Net_AcceptRxList@vmkernel#nover+0x157

Read more »