ESXi 5 Network Troubleshooting Commands

Check if a remote host is online and reachable.

~ # ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=58 time=13.701 ms
64 bytes from icmp_seq=1 ttl=58 time=10.176 ms
64 bytes from icmp_seq=2 ttl=58 time=9.055 ms

--- ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 9.055/10.977/13.701 ms

Ping from a specific VMkernel adapter.

~ # ping -I vmk1
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=58 time=9.991 ms
64 bytes from icmp_seq=1 ttl=58 time=9.270 ms

Verify end-to-end MTU size. If you have jumbo frames configured in your environment, this might be useful. The -d option disables fragmentation, the -s option sets the packet size. Decrement the packet size until the ping succeeds. Add 28 Byte to the largest possible packet size (IP and ICMP headers). The result is your MTU. For Jumbo frames, the expected packet size is 8972 Bytes. In the following example the MTU is 1500 (Ping possible with 1472 Bytes +28 Bytes header).

~ # ping -d -s 1473
PING ( 1473 data bytes
sendto() failed (Message too long)

~ # ping -d -s 1472
PING ( 1472 data bytes
1480 bytes from icmp_seq=0 ttl=64 time=0.885 ms
1480 bytes from icmp_seq=1 ttl=64 time=0.913 ms

Display routing table

~ # /usr/sbin/esxcfg-route -l
VMkernel Routes:
Network          Netmask          Gateway          Interface    Local Subnet     vmk0
default  vmk0

Track the route packets taken to a given host. This is applicable for routed connections only.

~ # traceroute
traceroute to (, 30 hops max, 40 byte packets
 1 c7200 ( 0.716 ms 0.623 ms 0.592 ms
 2 hh-ea7-i.HH.DE.NET.DTAG.DE ( 11.833 ms 11.297 ms 11.886 ms
 3 * ( 70.479 ms 70.370 ms
 4 ( 8.755 ms 8.301 ms 8.612 ms
 5 ( 7.652 ms 8.524 ms 8.343 ms

Display physical network adapters.

~ # esxcfg-nics -l
Name    PCI          Driver      Link Speed     Duplex MAC Address       MTU    Description
vmnic0  0000:07:00.0 tg3         Up   1000Mbps  Full   e4:11:5b:13:83:d3 1500   Broadcom Corporation NetXtreme BCM5723 Gigabit Ethernet
vmnic1  0000:03:00.0 igb         Down 0Mbps     Half   00:1b:21:93:b3:b0 1500   Intel Corporation 82576 Gigabit Network Connection
vmnic2  0000:03:00.1 igb         Down 0Mbps     Half   00:1b:21:93:b3:b1 1500   Intel Corporation 82576 Gigabit Network Connection
vmnic3  0000:04:00.0 igb         Down 0Mbps     Half   00:1b:21:93:b3:b2 1500   Intel Corporation 82576 Gigabit Network Connection
vmnic4  0000:04:00.1 igb         Up   1000Mbps  Full   00:1b:21:93:b3:b3 1500   Intel Corporation 82576 Gigabit Network Connection

Display physical network adapters including packet counters, ring parameters and driver information.

~ # /usr/lib/vmware/vm-support/bin/
NIC:  vmnic4

   Advertised Auto Negotiation: true
   Advertised Link Modes: 10baseT/Half, 10baseT/Full, 100baseT/Half, 100baseT/Full, 1000baseT/Full
   Auto Negotiation: true
   Cable Type: Twisted Pair
   Current Message Level: 7
   Driver Info:
         Bus Info: 0000:04:00.1
         Driver: igb
         Firmware Version: 1.2.1
   Link Detected: true
   Link Status: Up
   Name: vmnic4
   PHY Address: 1
   Pause Autonegotiate: true
   Pause RX: false
   Pause TX: false
   Supported Ports: TP
   Supports Auto Negotiation: true
   Supports Pause: true
   Supports Wakeon: false
   Transceiver: internal
   Wakeon: None
Ring parameters for vmnic4:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             256
RX Mini:        0
RX Jumbo:       0
TX:             256

NIC statistics for vmnic4:
   Packets received: 31935
   Packets sent: 4499
   Bytes received: 3651845
   Bytes sent: 276356
   Receive packets dropped: 0
   Transmit packets dropped: 0

Display VMkernel adapters.

~ # esxcfg-vmknic -l
Interface  Port Group/DVPort/Opaque Network        IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type                NetStack
vmk0       Management Network                      IPv4                 e4:11:5b:13:83:d3 1500    65535     true    STATIC              defaultTcpipStack
vmk0       Management Network                      IPv6      fe80::e611:5bff:fe13:83d3               64                              e4:11:5b:13:83:d3 1500    65535     true    STATIC, PREFERRED   defaultTcpipStack
vmk1       VMkernel                                IPv4                 00:50:56:69:62:af 1500    65535     true    DHCP                defaultTcpipStack
vmk1       VMkernel                                IPv6      fe80::250:56ff:fe69:62af                64                              00:50:56:69:62:af 1500    65535     true    STATIC, PREFERRED   defaultTcpipStack

Display ARP table

~ # esxcli network ip neighbor list
Neighbor         Mac Address        Vmknic     Expiry  State  Type
---------------  -----------------  ------  ---------  -----  -------   bc:5f:f4:45:31:22  vmk0     1189 sec         Unknown   00:1b:21:93:b9:a4  vmk0      272 sec         Unknown  74:31:70:4e:d7:be  vmk0     1197 sec         Unknown   (incomplete)       vmk0       -1 sec         Unknown
fe80::1          74:31:70:4e:d7:be  vmk0    85926 sec  Stale  Unknown

Verify that the host can reach ports on external server (ESXi Port Scanner). Actually it is the netcat command
In this example I am verifying that the vCenters https port, and iSCSI from an external storage is accessible.

~ # nc -z 443
Connection to 443 port [tcp/https] succeeded!
~ # nc -z 3260
Connection to 3260 port [tcp/*] succeeded!

Collect packet traces from a specific VMkernel interface.

 tcpdump-uw -i vmk0

Collect packet traces on a specific protocol. This command displays ICMP (ping) only.

~ # tcpdump-uw icmp
tcpdump-uw: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmk0, link-type EN10MB (Ethernet), capture size 96 bytes
19:53:31.339259 IP truncated-ip - 2 bytes missing! > esx1.virten.lab: ICMP echo request, id 237, seq 0, length 64
19:53:31.341207 IP truncated-ip - 2 bytes missing! esx1.virten.lab > ICMP echo reply, id 237, seq 0, length 64
19:53:32.342857 IP truncated-ip - 2 bytes missing! > esx1.virten.lab: ICMP echo request, id 237, seq 1, length 64
19:53:32.342918 IP truncated-ip - 2 bytes missing! esx1.virten.lab > ICMP echo reply, id 237, seq 1, length 64
19:53:33.348021 IP truncated-ip - 2 bytes missing! > esx1.virten.lab: ICMP echo request, id 237, seq 2, length 64
19:53:33.348103 IP truncated-ip - 2 bytes missing! esx1.virten.lab > ICMP echo reply, id 237, seq 2, length 64

6 packets captured
6 packets received by filter
0 packets dropped by kernel

Write tcpdump packet traces to a file for later analysis.

~ # tcpdump-uw -w dump.cap

Display active TCP/UDP connections.

~ # esxcli network ip connection list
Proto  Recv Q  Send Q  Local Address                    Foreign Address       State        World ID  CC Algo  World Name
-----  ------  ------  -------------------------------  --------------------  -----------  --------  -------  ---------------
tcp         0       0                ESTABLISHED     35309  newreno  hostd-worker
tcp         0     820                ESTABLISHED     33932  newreno  rhttpproxy-work
tcp         0       0                 ESTABLISHED     33934  newreno  rhttpproxy-work
tcp         0     795                 ESTABLISHED    406071  newreno  python
tcp         0       0        TIME_WAIT           0

Display virtual switch information

~ # /usr/sbin/esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         1536        8           128               1500    vmnic0,vmnic4

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  VMkernel              0        1           vmnic4,vmnic0
  VM Network            0        1           vmnic0,vmnic4
  Management Network    0        1           vmnic0,vmnic4

Verify SSL certificate information from remote hosts. This example checks the certificate from a vCenter Server.

~ # openssl s_client -connect
WARNING: can't open config file: /usr/ssl/openssl.cnf
depth=0 CN = vcsa6.virten.lab, C = US
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = vcsa6.virten.lab, C = US
verify error:num=27:certificate not trusted
verify return:1
depth=0 CN = vcsa6.virten.lab, C = US
verify error:num=21:unable to verify the first certificate
verify return:1
Certificate chain
 0 s:/CN=vcsa6.virten.lab/C=US
   i:/CN=CA, dc=vsphere,dc=local/C=US/O=vcsa6.virten.lab
Server certificate
issuer=/CN=CA, dc=vsphere,dc=local/C=US/O=vcsa6.virten.lab
No client certificate CA names sent
SSL handshake has read 996 bytes and written 623 bytes
New, TLSv1/SSLv3, Cipher is AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
    Protocol  : TLSv1.2
    Cipher    : AES256-GCM-SHA384
    Master-Key: B289EEF91BFF7572C9641A6735E1B2A8E750C9DAA7FE3DD9510FA4FCCC3D0FE200AFAB967C71E9370FE63EBA6012B5BF
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1423597583
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)
bad select 4
  1. Hi,

    there's a way to force the vmkping to go out a specific physical nic? For example: if I have a vDS with two vmnics, how can "vmkping/force" a vmkernel interface to use one of these 2 vmnics?


    • You can't force vmkping to a specific physical nic. You can only force it to use a vmkernel adapter (ping -I vmk1 A vmkernel adapter is bound to a physical interface and will (depending on the failover policy) always use that adapter. As a workaround, you could bind the dvPortgroup containing the vmkadapter to a specific dvUplink (containing your vmnic). (dvPortgroup Settings > Policies > Teaming and Failover). This will force your ping (and all other traffic!) to a specific physical interface.

  2. when using traceroute command to IPv6 storage address
    traceroute ipv6 2a00:da9:a:xxxx::xxxx

    I get "bad value for packet lenght"

    tried traceroute ipv6 and traceroute address

    normal traceroute gives "unknown host"

    Host is pingable. I am puzzled.

Leave a Comment

NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Trackbacks and Pingbacks: