In the Getting Started with NSX Advanced Load Balancer Integration in VMware Cloud Director 10.3 Guide, I've explained how to enable "Load Balancing as a Service" in VCD with dedicated Service Engines. With this Service Engine deployment model, each Edge Gateway is statically assigned to a dedicated NSX-ALB Service Engine Group. That means, for each EGW you create in VCD, you have to create a Service Engine Groups, which consists of multiple Service Engines (Virtual Machines).
Service Engine Groups can also be deployed in a shared model. Shared Service Engine groups can be assigned to multiple Edge Gateways. In this deployment model, a single Service Engine (Virtual Machine) can handle traffic for multiple customers. For obvious security reasons, and to prevent problems with overlapping networks, VRFs are used inside the SE to fully separate the data traffic.
This article explains how to use Shared Service Engine Groups in VMware Cloud Director 10.3.
- NSX-ALB integration in VMware Cloud Director is fully configured, as explained in this article.
- Versions used in this article: Cloud Director 10.3, NSX-T 18.104.22.168 and NSX-ALB 21.1.1
Shared Service Groups vs. Dedicated Service Groups
Whether a Service Engine Group (SEG) is shared or dedicated is configured when the SEG is imported into VCD. From NSX-ALB perspective, there is no difference in the SEG configuration. NSX-ALB uses a separate VRF instance for each Edge Gateway that is used in the NSX-T Cloud. It does not matter whether the Service Engine is dedicated or shared, the data interface is always in a VRF.
The following picture shows a Shared Service Engine Group with two Service Engine that serves two different VMware Cloud Director Tenants. As you can see from the picture, the SE Data Network is always 192.168.255.0/25 and both customers are using 192.168.0.0/24 as their Routed VDC Network. This is possible because Service Engines are using a separate Routing Instance (VRF) for each EGW.
The following picture shows a Dedicated Service Engine Group which can only be used on a single Edge Gateway only. The VRF for the Service Engine Data Network exists but is omitted in the picture.
Note: If a tenant wants to use 192.168.255.0/25, you can change the SE Network in the EGW LB config (Edge Gateway > Load Balancer > General Settings > Service Network Specification).
Step 1 - Create Service Engine Group in NSX-ALB
Open NSX-ALB and navigate to Infrastructure > Cloud Resources > Service Engine Groups and select the NSX-T Cloud at the top of the page. Make sure to not create the SEG in the Default Cloud. Press CREATE to create a new Service Engine Group.
- Shared Engine Group Name: Display Name for the SEG
- High Availability Mode: All 3 modes are supported with VCD. I recommend using one of the Elastic HA modes. With Active/Active, each virtual service is placed on more than one SE. With N+M, Virtual Services are typically placed on just one SE, and failover happens when a SE fails. (See: AVI documentation)
- VS Placement across SEs: In Compact mode, multiple virtual services are deployed on existing Service Engines, until the "Virtual Services per SE" limit is reached. In Distributed mode, a new SE is deployed for each virtual service until the "Max Number of SEs" is reached. When the limit is reached, virtual services are deployed on existing SEs.
- Virtual Services per Service Engine / Max Number of Service Engines: These two numbers multiplied resulting in the total number of virtual services that can be deployed in the SEG. 10 x 10 = 100 Services. The available number of virtual services can be assigned to tenants in VCD.
If necessary, you can specify placement options in the Advanced tab. In this example, I've limited the deployment to the "NUCluster" Cluster and "bort-red" datastores. If not specified, AVI will place Service Enginges anywhere in the vCenter.
Step 2 - Import Service Engine Group to Cloud Director (VCD Admin Part)
Login to Cloud Director as System Admin and navigate to Resources > Infrastructure Resources > NSX-ALB > Service Engine Groups. Press ADD to import the previously created SEG.
- NSX-T Cloud: nsx1.virten.lab (Requires existing NSX-ALB integration)
- Reservation Mode: Shared
- Name: shared-seg-001 (Can differ from the name used in NSX-ALB. This is the name that the tenant can see)
- Select the SEG created in Step 1 from Available Service Engine Groups
Press ADD to finish the configuration. The service group can now be assigned to multiple Edge Gateways. Navigate to Resources > Cloud Resources > Edge Gateways and click on the tenant's EGW Name to enter the configuration. From there, navigate to Load Balancer > General Settings, press EDIT and enable the Load Balancer. Now navigate to Load Balancer > Service Engine Groups and press ADD.
This is the part where the shared SEG differs from the dedicated SEG. You can limit the amount of Virtual Services that the customer can create in the SEG. The available capacity (100) is given by the number of Virtual Services per Service Engine and Max Number of Service Engines, configured in the SEG configuration in NSX-ALB. Go ahead and limit the user to a maximum of 10 Virtual Services.
The tenant can now create Pools and Virtual Services on his Edge Gateway. The following screen shows the Service Engine status after deploying the first Service. The tenant can deploy up to 10 Services.