Florian Grehl

Create Virtual Machines in vSphere with Tanzu using kubectl

This article explains how you can create Virtual Machines in Kubernetes Namespaces in vSphere with Tanzu. The deployment of Virtual Machines in Kubernetes namespaces using kubectl was shown in demonstrations but is currently (as of vSphere 7.0 U2) not supported. Only with third-party integrations like TKG, it is possible to create Virtual Machines by leveraging the vmoperator.

With the kubernetes-admin, accessible from the SupervisorControlPlane VM, you can create Virtual Machines today.

Please keep in mind that this is not officially supported by VMware.

Read More »Create Virtual Machines in vSphere with Tanzu using kubectl

Deploy NSX-T Edge VM SSH Keys with Ansible

While working with NSX-T, there are many reasons to access edge appliances using SSH. Most troubleshooting options are only available using nsxcli on the appliance itself. During the deployment, each appliance has 3 user account: root, admin, and audit. Alle Accounts are configured with password-based authentication. In a previous article, I've already described how to deploy SSH Keys using nsxcli, which allows a secure and comfortable authentication method. In this article, I'm explaining how to use ansible to deploy SSH public keys to NSX-T Edges. This option allows you to easily manage keys on a large platform.

Read More »Deploy NSX-T Edge VM SSH Keys with Ansible

VMware ESXi 7.0 Update 2 on Intel NUC

VMware vSphere ESXi 7.0 Update 2 has been released this week and before you start to deploy it to production, you want to evaluate it in your testing environment or homelab. If you have Intel NUCs you should always be very careful when updating to new ESXi releases as there might be issues. Please always keep in mind that this is not an officially supported platform.

Within the 7.0 releases, there are many issues with consumer network adapters, like the deprecation of VMKlinux drivers and thus the missing support for Realtek NICs, and the up and downs with the ne1000 driver.

To be on the safe side, I'm doing a quick checkup on which NUCs are safe to update and what considerations you have to take before installing the update. Also, I'm quickly explaining the options to workaround the crypto64.efi issue.

Read More »VMware ESXi 7.0 Update 2 on Intel NUC

Error when connecting Virtual Machine to NSX-T Segments

When you try to connect an NSX-T based Segment to a virtual machine, the task fails with the following error message:

Reconfigure virtual machine - An error occurred during host configuration

In the nsx logfile on the ESXi host where the VM is located, the following error is displayed:

/var/log/nsx-syslog.log
2021-03-13T19:00:36Z nsx-opsagent[527252]: NSX 527252 - [nsx@6876 comp="nsx-esx" subcomp="opsagent" s2comp="nsxa" tid="527596" level="ERROR" errorCode="MPA44211"] [PortOp] Failed to create port 780b915d-1479-4eed-8e29-2364d9563f95 with VIF f3f605f2-38a1-4263-bbbd-81b189077f69 because DVS id is not found by transport-zone id 1b3a2f36-bfd1-443e-a0f6-4de01abc963e
2021-03-13T19:00:36Z nsx-opsagent[527252]: NSX 527252 - [nsx@6876 comp="nsx-esx" subcomp="opsagent" s2comp="nsxa" tid="527596" level="ERROR" errorCode="MPA42001"] [CreateLocalDvPort] createPort(uuid=780b915d-1479-4eed-8e29-2364d9563f95, zone=1b3a2f36-bfd1-443e-a0f6-4de01abc963e) failed: Failed to create port 780b915d-1479-4eed-8e29-2364d9563f95 with VIF f3f605f2-38a1-4263-bbbd-81b189077f69 because DVS id is not found by transport-zone id 1b3a2f36-bfd1-443e-a0f6-4de01abc963e

 

Read More »Error when connecting Virtual Machine to NSX-T Segments

Failed to load crypto64.efi - ESXi 7.0 U2 Upgrade Error

When you try to upgrade your ESXi host to the latest 7.0 U2 release using either the predefined update baselines or by using esxcli with the upgrade bundle, your ESXi host might fail to reboot with the following error message.

Loading /boot.cfg
Failed to load crypto64.efi
Fatal error: 15 (Not found)

The error can not be solved with the Shift+R method to restore the previous ESXi version. VMware is aware of the problem and has already removed the update bundle (VMware-ESXi-7.0U2-17630552-depot.zip) and Image Profile (ESXi-7.0.2-17630552-standard) from their repository. Currently, you only have two options to upgrade to ESXi 7.0 Update 2. If you already ran into the "Failed to load crypto64.efi" error, you have to take option 1, which will fix the error.

[Update 2021-03-13] - VMware has also disabled the image profile for 7.0.2. If you try an online update using ESXCLI or want to create a custom image using Imagebuilder, you get the following error:

[NoMatchError] No image profile found with name 'ESXi-7.0.2-17630552-standard' id = ESXi-7.0.2-17630552-standard Please refer to the log file for more details.

Read More »Failed to load crypto64.efi - ESXi 7.0 U2 Upgrade Error

How to Backup and Restore NSX-T

NSX-T is a critical infrastructure component and it is crucial to have a working backup and restore plan. With complex products, the backup and restore strategy gets more complicated. When working with Virtual Machines, the backup is usually done with VMware Snapshots, which is super convenient. Unfortunately, with the complexity of NSX-T which has many components like clustered Managers, Transport Nodes, and ESXi Kernel Modules, you can't use snapshots as a backup strategy.

This article provides an overview of how to backup NSX-T, and how the restore is done properly.

Read More »How to Backup and Restore NSX-T

vSphere with Tanzu - SupervisorControlPlaneVM Excessive Disk WRITE IO

After deploying the latest version of VMware vSphere with Tanzu (vCenter Server 7.0 U1d / v1.18.2-vsc0.0.7-17449972), I noticed that the Virtual Machines running the Control Plane (SupervisorControlPlaneVM) had a constant disk write IO of 15 MB/s with over 3000 IOPS. This was something I didn't see in previous versions and as this is a completely new setup with no namespaces created yet, there must be an issue.

After troubleshooting the Supervisor Control Plane, it turned out that the problem was caused by fluent-bit, which is the Log processor used by Kubernetes. The log was constantly spammed with debugging messages. Reducing the log level solved the problem for me.

[Update: 2021-03-14 - The problem is not resolved in vSphere 7.0 Update 2]

Read More »vSphere with Tanzu - SupervisorControlPlaneVM Excessive Disk WRITE IO

11th Gen Intel NUC - Which is the best candidate to run ESXi?

Intel has finally announced their 11th Generation NUCs. For the first time, all three product lines are announced at the same time. The NUC series is very popular to be used in homelabs or for running VMware ESXi. They are small, silent, transportable, and have very low power consumption.

In this article, I'm going to take a look at the 3 different product lines and how they compare to each other and previous NUCs.

Read More »11th Gen Intel NUC - Which is the best candidate to run ESXi?