Skip to content

k8s

Deploy Container Service Extension (CSE 3.0) in VMware Cloud Director 10.2

With the release of Cloud Director 10.2, the Container Service Extension 3.0 has been released. With CSE 3.0 you can extend your cloud offering by providing Kubernetes as a Service. Customers can create and manage their own K8s clusters directly in the VMware Cloud Director portal.

I've already described how to deploy vSphere with Tanzu based Kubernetes Clusters in VCD. CSE 3.0 with the "Native K8s Runtime" is is a neat alternative that allows you to deploy K8s directly into the customer's Organization networks, which is currently not possible with Tanzu.

This article explains how to integrate CSE 3.0 in VMware Cloud Director 10.2.

Read More »Deploy Container Service Extension (CSE 3.0) in VMware Cloud Director 10.2

Tanzu Kubernetes Licensing in vSphere 7.0 Update 1

With the release of vSphere 7.0 Update 1, VMware introduced a new licensing model for its Tanzu Kubernetes integration. Basically, the licensing has been changed from an ESXi-Host license to a Cluster license that looks familiar to the vSAN license which is in place for a couple of years. The change does only affect how you have to apply the license. The entity to pay for is still a physical CPU.

In vSphere 7.0 GA, the license required to enable Kubernetes (aka. "Workload Management") was an add-on license for ESXi Hosts named "vSphere 7 Enterprise Plus with Kubernetes". With the introduction of vSphere 7.0 Update 1, which is also referred to as 7.0.1, "vSphere add-on for Kubernetes" has been rebranded and split into 4 licenses Tanzu Basic, Tanzu Standard, Tanzu

Read More »Tanzu Kubernetes Licensing in vSphere 7.0 Update 1

Quick Tip: Reset Tanzu SupervisorControlPlaneVM Alarms

When you are working with the Kubernetes Integration in vSphere 7.0, you might come into the situation where the SupervisorControlPlaneVM has an active alarm. Those Virtual Machines are deployed and controlled by the WCP Agent and even as an Administrator, you are not allowed to touch those objects.
You can't power then off, reboot, or migrate them using vMotion. The problem is that you can't even clear alarms. One alarm I recently had was the "vSphere HA virtual machine failover failed" alarm, which you usually see when the ESXi hostd crashed, but the Virtual Machines are still running.Read More »Quick Tip: Reset Tanzu SupervisorControlPlaneVM Alarms

Change TKG Cluster Service and Pod CIDR in Cloud Director 10.2

A major problem when deploying "vSphere with Tanzu" Clusters in VMware Cloud Director 10.2 is that the defaults for TKG Clusters are overlapping with the defaults for the Supervisor Cluster configured in vCenter Server during the Workload Management enablement.

When you deploy a Kubernetes Cluster using the new Container Extension in VCD 10.2, it deploys the cluster in a namespace on top of the Supervisor Cluster in the vCenter Server. The Supervisor Clusters IP address ranges for the Ingress CIDRs and Services CIDR must not overlap with IP addresses 10.96.0.0/12 and 192.168.0.0/16, which is the default for TKG Clusters. Unfortunately, 10.96.0.0 is also the default when enabling workload management so the deployment will fail when you stick to the defaults. The following error message is displayed when you have overlapping networks:

spec.settings.network.pods.cidrBlocks intersects with the network range of the external ip pools in network provider's configuration
spec.settings.network.pods.cidrBlocks intersects with the network range of the external ip pools LB in network provider's configuration

This article explains a workaround that you can apply when deleting and reconfiguring the Namespace Management with non-overlapping addresses is not an option.

Read More »Change TKG Cluster Service and Pod CIDR in Cloud Director 10.2

Troubleshooting "vSphere with Tanzu" Integration in VCD 10.2

During my first attempts to integrate "vSphere with Tanzu" into VMware Cloud Director 10.2, I had a couple of issues. The integration just wasn't as smooth as I expected and many configuration errors are not mitigated in the GUI. Also, there are a lot of prerequisites to strictly follow.

In this article, I'm going through the issues I had during the deployment and how to solve them.Read More »Troubleshooting "vSphere with Tanzu" Integration in VCD 10.2

Configure "vSphere with Tanzu" in VMware Cloud Director 10.2

With the release of Cloud Director 10.2, you can now integrate "vSphere with Tanzu" Kubernetes Clusters into VMware Cloud Director. That enabled you to create a self-service platform for Kubernetes Clusters that are backed by the Kubernetes integration in vSphere 7.0.

This article explains how to integrate vSphere with Tanzu in VMware Cloud Director 10.2

Read More »Configure "vSphere with Tanzu" in VMware Cloud Director 10.2

vSphere with Kubernetes - Which Supervisor Cluster Settings can be edited?

When you want to deploy Kubernetes on vSphere 7 it is crucial to plan the configuration thoroughly prior to enabling Workload Management. Many of the configuration parameters entered during the Workload Management wizard can not be changed after the deployment.

The following table show which settings can be changed after the initial deployment:

Read More »vSphere with Kubernetes - Which Supervisor Cluster Settings can be edited?

vSphere with Kubernetes Supports Multiple Tier-0 Gateways

During my first vSphere with Kubernetes tests, I had an issue where I was not able to activate Workload Management (Kubernetes) because it discovered multiple Tier-0 gateways. The configuration I used was vSphere 7.0 GA and an NSX-T 3.0 backed N-VDS. I had a previously configured Edge Cluster / Tier-0 Gateway for existing workloads and configured a new Edge Cluster / Tier-0 for Kubernetes.

In the Workload Management Wizard, no Cluster was compatible so I was forced to use the previously configured Tier-0 with some routing workarounds. The error message in wcpsvc.log stated "[...]has more than one tier0 gateway[...]".

Today I tried to find a solution and noticed that there was an update to the official Kubernetes Guide:

Read More »vSphere with Kubernetes Supports Multiple Tier-0 Gateways

VMware vSphere with Kubernetes Guide Part 7 - Octant and Lens

This is the last part of my "VMware vSphere with Kubernetes" Guide. In this article, I'm going to give you two tools that will help you to get a better understanding of Kubernetes features. Both tools, Octant and Lens, are free and Open Source. The main difference is that Octant is browser-based and Lens is an Application.

Read More »VMware vSphere with Kubernetes Guide Part 7 - Octant and Lens

VMware vSphere with Kubernetes Guide Part 6 - Tanzu Kubernetes Cluster

This is Part 6 of my "VMware vSphere with Kubernetes" Guide. In this article, I'm going to deploy a Tanzu Kubernetes Cluster (TKC). A TKC is a fully-featured version of the open-source Kubernetes container platform. You can provision and operate Tanzu Kubernetes clusters on top of the Supervisor Cluster.

Read More »VMware vSphere with Kubernetes Guide Part 6 - Tanzu Kubernetes Cluster