VMware NSX-T 3.1 Edge Node Sizing

Edge Nodes in NSX-T 3.1 are available as Virtual Machines and Bare Metal Edges. When you deploy a Virtual Edge Node using the embedded deployment function in NSX-T, you can choose between 4 sizes - Small, Medium, Large and Extra Large. In this article, I'm trying to collect information about the different sizing options, what they are intended for and how to resize Edge Nodes.

NSX-T Edge VM Sizing Options

NSX-T Edge VM sizing options with their recommended bandwidth range and service usage as a table.

MemoryCPUDiskBandwidthNAT/FirewallL4 LBL7 LBMulti-Gbps
L7 LB / VPN
Small4 GB2200 GB< 2GbpsPoC Only
Medium8 GB4200 GB2 GbpsYESYESNONO
Large32 GB8200 GB2-10 GbpsYESYESYESNO
X-Large64 GB16200 GB> 10 GbpsYESYESYESYES

Description

  • NSX Edge Small - Proof-of-concept deployments only.
  • NSX Edge Medium - Suitable when only L2 through L4 features such as NAT, routing, L4 firewall, L4 load balancer are required and the total throughput requirement is less than 2 Gbps.
  • NSX Edge Large - Suitable when only L2 through L4 features such as NAT, routing, L4 firewall, L4 load balancer are required and the total throughput is 2 ~ 10 Gbps. It is also suitable when L7 load balancer, for example, SSL offload is required.
  • NSX Edge Extra Large - Suitable when the total throughput required is multiple Gbps for L7 load balancer and VPN.

Reference: NSX Edge VM System Requirements

NSX-T Bare Metal Edge Sizing

A Bare Metal Edge is typically deployed, where higher performance and faster North-South convergence is desired. The Operating System used for NSX-T Edges 3.1 Edges is Ubuntu 18.04, which is important when you want to purchase Servers. Make sure to use Ubuntu certified hardware.

MemoryCPUDisk
Minimum Requirements32 GB8200 GB
Recommended256 GB24200 GB

Reference: NSX Edge Bare Metal Requirements

 

NSX-T Load Balancer Sizing

When you want to use the NSX-T embedded Load Balancer Service (Not NSX-ALB / AVI), which runs on top of a Tier-1 Gateway, you can select between Small, Medium, Large and X-Large. The LB Service size dictates how many Virtual Servers, Pools and Pool Members are supported.

Virtual ServersPoolsPool Members
Small2060300
Medium1003002000
Large100030007500
X-Large2000400010000

These limits are Hard Limits. NSX-T prevents you from adding more Virtual Servers with the following error message:

Exceed maximum number of virtual servers on the Load balancer service. The maximum size of virtual servers for SMALL load balancer service form factor is 20, current size of virtual servers is 21. (code 502090)

The limit of Virtual Services per Edge Node is also limited. The following table shows which LB Instances can be deployed with each available Edge Size:

Pool MembersSmall LBMedium LBLarge LBX-Large LB
Small01000
Medium200010100
Large750040410
X-Large1000080821
Bare-Metal3000075075189

These limits are Hard Limits. NSX-T prevents you from adding more LB Services with the following error message:

There is no available capacity on edge node [NODE Path] to deploy a LARGE load balancer service, the form factor of this edge node is LARGE_VIRTUAL_MACHINE, the existed load balancer services on the edge node are [LIST].

You can get a capacity report using the NSX-T API with either Manager or Policy mode. With Manager Mode, you just need the Edge VM UUID (System > Fabric > Nodes > Edge Transport Nodes > Edge > ID). The report shows the current and remaining capacity:

# curl -k -u admin:password https://[NSX-Manager]/api/v1/loadbalancer/usage-per-node/76d7e438-c901-415a-b3b9-ffb854ba31a3
{
  "form_factor" : "LARGE_VIRTUAL_MACHINE",
  "edge_cluster_id" : "5fa37405-5c9b-47f3-aec8-81f0eb507519",
  "current_credit_number" : 2,
  "remaining_credit_number" : 38,
  "usage_percentage" : 5.0,
  "severity" : "GREEN",
  "current_pool_members" : 9,
  "current_virtual_servers" : 22,
  "current_pools" : 5,
  "current_small_load_balancer_services" : 2,
  "current_medium_load_balancer_services" : 0,
  "current_large_load_balancer_services" : 0,
  "current_xlarge_load_balancer_services" : 0,
  "remaining_small_load_balancer_services" : 38,
  "remaining_medium_load_balancer_services" : 3,
  "remaining_large_load_balancer_services" : 0,
  "remaining_xlarge_load_balancer_services" : 0,
  "remaining_pool_members" : 7491,
  "type" : "LbEdgeNodeUsage",
  "node_id" : "76d7e438-c901-415a-b3b9-ffb854ba31a3"
}

With Policy Mode, the call is slightly more complex as you have to provide an enforcement point, which is the edge node path including the edge cluster. Example (The UUID used here is the Edge Cluster UUID):

# curl -k -u admin:password https://[NSX-Manager]/policy/api/v1/infra/lb-node-usage?node_path=/infra/sites/default/enforcement-points/default/edge-clusters/5fa37405-5c9b-47f3-aec8-81f0eb507519/edge-nodes/0
{
  "form_factor" : "LARGE_VIRTUAL_MACHINE",
  "edge_cluster_path" : "/infra/sites/default/enforcement-points/default/edge-clusters/5fa37405-5c9b-47f3-aec8-81f0eb507519",
  "current_load_balancer_credits" : 2,
  "load_balancer_credit_capacity" : 40,
  "usage_percentage" : 5.0,
  "severity" : "GREEN",
  "current_pool_member_count" : 9,
  "current_virtual_server_count" : 22,
  "current_pool_count" : 5,
  "pool_member_capacity" : 7500,
  "current_small_load_balancer_count" : 2,
  "current_medium_load_balancer_count" : 0,
  "current_large_load_balancer_count" : 0,
  "current_xlarge_load_balancer_count" : 0,
  "remaining_small_load_balancer_count" : 38,
  "remaining_medium_load_balancer_count" : 3,
  "remaining_large_load_balancer_count" : 0,
  "remaining_xlarge_load_balancer_count" : 0,
  "resource_type" : "LBEdgeNodeUsage",
  "node_path" : "/infra/sites/default/enforcement-points/default/edge-clusters/5fa37405-5c9b-47f3-aec8-81f0eb507519/edge-nodes/0"

 

The informal from both API calls are the same. You can see the current and remaining LB counter. From the call, you can also learn that NSX-T uses "Credits" to manage the Load Balancer capacity. For each service, the following amount of credits are accounted on the Edge VM:

Small Load Balancer Service1 Credit
Medium Load Balancer Service10 Credits
Large Load Balancer Service40 Credits
X-Large Load Balancer Service80 Credits

The available Credits per Edge VM Size is basically the same:

Small Edge VM1 Credit
Medium Edge VM10 Credits
Large Edge VM40 Credits
X-Large Edge VM80 Credits

Resize NSX-T Load Balancer Service Size

The Load Balancer Service size can be easily changed after the deployment. Please keep in mind that changing and redeploying the service will close all active sessions. There is also a warning when you try to change the size (Networking > Network Services > Load Balancer > Load Balancers > Edit > Size).

 

Resize NSX-T Edge VMs

There is no official documentation on how to change the size of Edge VMs. If you strictly follow the documentation, the only valid option is to deploy a new Edge VM with the new size and use the "Replace Edge Cluster Member" function. Procedure:

  1. Deploy a new Edge VM (System > Fabric > Nodes > Edge Transport Nodes > Add Edge Node)
  2. Place the old Edge VM into Maintenance Mode (System > Fabric > Nodes > Edge Transport Nodes > Mark the Node > Actions > Enter NSX Maintenance Mode)
  3. Replace the old Edge VM (System > Fabric > Nodes > Edge Clusters > Mark the Edge Cluster > Actions > Replace Edge Cluster Member)
  4. Select the old and new Edge VM and press SAVE

Reference: Replace an NSX Edge Transport Node Using the NSX Manager UI

For non-production environments, you can also shut down the Edge VM and just change the Virtual Machine Size from vCenter. I've done that multiple times and never had any issues. However, I've not found any information that this is a supported procedure. After changing the size, using the lb-node-usage API call explained above, you can see that the Edge VM is indeed able to detect the new Form Factor.

 

 

2 thoughts on “VMware NSX-T 3.1 Edge Node Sizing”

    1. Hard to tell because it does not solely depend on throughput. The type of traffic (Concurrent Sessions, Packet Size) and of course the cryptographic configuration is also important. According to VMware, you should use the X-Large for "Multiple Gbps VPN". With a single large stream, you can have 3-4 GBps even with a Medium Edge.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.