Identify Virtual Machine Locks & Find ESXi Hosts by its MAC Address

I recently had an issues where a virtual machine crashed. VM Monitoring (VM HA) tried to restart it, but did not succeed. The virtual machine was greyed out in the inventory and could not be started because it was locked. Unfortunately, it was not possible to identify which ESXi host holds the lock. This post explains how to quickly identify which server is blocking the Virtual Machine.

Connect to any ESXi host with access to the datastore where the virtual machine is located.
Verify that the VM configuration file is locked, and idenity the owner

root@esx03:/vmfs/volumes/ds1/vma $ vmkfstools -D vma.vmx
Lock [type 10c00001 offset 200423424 v 59, hb offset 3874816
gen 3201, mode 1, owner 9f34545b-08d89e8d-6223-e4115b1383d3 mtime 369317
num 0 gblnum 0 gblgen 0 gblbrk 0]
Addr <4, 447, 15>, gen 24, links 1, type reg, flags 0, uid 0, gid 0, mode 100755
len 4548, nb 1 tbz 0, cow 0, newSinceEpoch 1, zla 2, bs 8192

The last part of the owner ID is the physical MAC address. It is usually the physical MAC address of the vmnic0 interface.

e4115b1383d3 = e4:11:5b:13:83:d3

Now you have to find the ESXi host that has this MAC address. This might be challenging in a large environment. I usually use a PowerCLI one-liner, or a SSH loop:

PowerCLI Script to find an ESXi host by its MAC address

PS C:\> Get-VMHostNetworkAdapter |? {$_.Mac -eq "e4:11:5b:13:83:d3"} | select VMhost, Name, Mac

VMHost                Name        Mac 
------                ----        --- 
esx01.virten.local    vmnic0      e4:11:5b:13:83:d3

SSH Loop to find and ESXi host by its MAC address

$ for i in {1..03}; do hostn=`printf "esx%02d.virten.local" $i`;echo $hostn; ssh root@$hostn 'esxcli network nic list |grep e4:11:5b:13:83:d3'; done
esx01.virten.local
vmnic0 0000:003:00.2 tg3 Down 0 Half e4:11:5b:13:83:d3 1500 Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet 
esx02.virten.local
esx03.virten.local

Kill the Virtual Machine to release the lock.

root@esx03:~ $ esxcli vm process list
vma.virten.local
 World ID: 5802380
 Process ID: 0
 VMX Cartel ID: 5802373
 UUID: 42 05 8d a2 2e 22 e6 45-16 68 0a 99 9a bd a3 f4
 Display Name: vma.virten.local
 Config File: /vmfs/volumes/ds1/vma/vma.vmx

 root@esx03:~ $ esxcli vm process kill -t force -w 5802380

You should be able to power on the Virtual Machine now. Perhaps you have to remove and add it back to the vCenter inventory.