vSphere with Tanzu 7.0U2a - TKC Deployment fails with VirtualMachineClassBindingNotFound

Since the latest update of vSphere with Tanzu to version 7.0 U2a, the deployment of Tanzu Kubernetes Clusters fails with the following condition:

  Conditions:
    Last Transition Time:  2021-05-05T18:19:10Z
    Message:               1 of 2 completed
    Reason:                VirtualMachineClassBindingNotFound @ Machine/tkc-dev-control-plane-wxd57
    Severity:              Error
    Status:                False
    Message:               0/1 Control Plane Node(s) healthy. 0/2 Worker Node(s) healthy
Events:
  Type    Reason        Age    From                                                                                             Message
  ----    ------        ----   ----                                                                                             -------
  Normal  PhaseChanged  7m22s  vmware-system-tkg/vmware-system-tkg-controller-manager/tanzukubernetescluster-status-controller  cluster changes from creating phase to failed phase

The problem seems to be related to the newly introduced VM Service. In previous versions, all Virtual Machine classes were automatically available for all namespaces. With the new VM Service, you can now create custom classes and assign them to namespaces. When a VirtualMachineClass is added to a namespace (Using the VM Service Card), a VirtualMachineClassBinding is created in the developer's namespace. This binding is not only required for Virtual Machines created by VM Service but also to deploy TKC Clusters.

When you create a new Namespace in vSphere 7.0 U2a, the namespace is created with no VirtualMachineClassBindings.

# kubectl get virtualmachineclassbindings
No resources found in biernot namespace.

After adding a VM Class to the Namespace using the vSphere Client, the binding is created and you can deploy TKG Clusters.

 

# kubectl get virtualmachineclassbindings
NAME                 VIRTUALMACHINECLASS  AGE
best-effort-xsmall   best-effort-xsmall   15m

# kubectl get virtualmachineclasses
NAME                  CPU   MEMORY   AGE
best-effort-2xlarge   8     64Gi     10d
best-effort-4xlarge   16    128Gi    10d
best-effort-8xlarge   32    128Gi    10d
best-effort-large     4     16Gi     10d
best-effort-medium    2     8Gi      10d
best-effort-small     2     4Gi      10d
best-effort-xlarge    4     32Gi     10d
best-effort-xsmall    2     2Gi      10d
guaranteed-2xlarge    8     64Gi     10d
guaranteed-4xlarge    16    128Gi    10d
guaranteed-8xlarge    32    128Gi    10d
guaranteed-large      4     16Gi     10d
guaranteed-medium     2     8Gi      10d
guaranteed-small      2     4Gi      10d
guaranteed-xlarge     4     32Gi     10d
guaranteed-xsmall     2     2Gi      10d


7 thoughts on “vSphere with Tanzu 7.0U2a - TKC Deployment fails with VirtualMachineClassBindingNotFound”

  1. Hi Florian, all

    Thanks for the instructions, I ran into the same failed TKC cluster deployment issue with my greenfield infrastructure running on vSphere & ESXi: 7.0U2a, NSX-T 3.2.1.
    The difference I experienced was, that the "k get virtualmachineclasses" did replay correctly even before assigning a VM Service Class the the vSphere Namespace. The " k describe tanzukubernetesclusters" provided the same Reason as you've described in the blog. Assigning a VM Service Class to the vSphere Namespace solved it and TKC cluster is deployed successfully.

    Thanks for the blog and cheers, CHrigi

    1. virtualmachineclasses are always available. The missing, and for the deployment required, part is the namespaces-scoped virtualmachineclassbindings

      1. then, the blog contains a typo...
        You've wrote:
        When you create a new Namespace in vSphere 7.0 U2a, the namespace is created with no
        VirtualMachineClassBindings.
        # kubectl get virtualmachineclasses
        No resources found in biernot namespace.

        But it should be this command "kubectl get virtualmachineclassbindings" instead

  2. I have a doubt. How fix it in an automatic way? Look is no make sense configure a policy in icloud director and go to vSphere to finish the work.

  3. Hi,
    is there any solution to automate this approach? Look, it doesn't make sense to create a namespace (kubernetes policy) in Vcloud Director and adjust this procedure in vSphere. IMO the right thing is for vCloud Director to make all parts of the procedure transparent to the operator.
    Regards

Leave a Reply to CHrigi Cancel reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.