In ESXi 6.5, there are some changes concerning devices connected with USB. The legacy drivers, including xhci, ehci-hcd, usb-uhci, and usb-storage have been replaced with a single USB driver named vmkusb. The new driver has some implications if you are trying to use USB devices like USB sticks or external hard disks as VMFS formatted datastore.
Some people have reported that they have issues with USB Datastores since ESXi 6.5. I've tried to reproduce and fix those problems. This post explains the changes in the new version and how to create VMFS 5 or VMFS6 formatted USB devices as datastore on your ESXi host.
Please remember that using USB Datastores is not supported by VMware so if you have problems, do not open service requests for that.
Device Naming Changes in ESXi 6.5
Which I noticed first was that the devices naming in 6.5 have been changed. In ESXi 6.0, all USB devices are recognized as mpx.vmhbaXX while in 6.5 some are listed as naa.XXX. Here is an example of a Samsung M.2 flash drive connected with a USB3 adapter:
# ESXi 6.0u2 -rw------- 1 root root 128035676160 Nov 27 15:57 naa.5000000000000001 # ESXi 6.5 -rw------- 1 root root 128035676160 Nov 27 18:16 mpx.vmhba36:C0:T0:L0
However, not all devices are affected. My 16GB flash drives shows up with the old mpx.vmhba naming on both versions:
# ESXi 6.0u2 -rw------- 1 root root 16022241280 Nov 27 16:02 mpx.vmhba34:C0:T0:L0 # ESXi 6.5 -rw------- 1 root root 16022241280 Nov 27 16:18 mpx.vmhba34:C0:T0:L0
The new naming does not have an impact on USB datastores, despite it might happen that you overlook the USB device when checking the /dev/disks/ directory. You can still partition and format USB devices with VMFS like in previous versions, just use the new name.
Existing Datastores are not mounted automatically
A common problem which I've heard multiple times, but only seen since ESXi 6.5, is that when you connect a flash drive that already contains a VMFS datastore, the datastore does not show up. You can see the device in /dev/disks but the datastore remains offline. The vmkernel.log hints the problem:
2016-11-27T10:25:05.885Z cpu0:65593)VMWARE SCSI Id: Id for vmhba34:C0:T0:L0 0x50 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x41 0x53 0x32 0x31 0x31 0x35 2016-11-27T10:25:05.885Z cpu0:65593)ScsiDeviceIO: 9120: Get VPD 80 Inquiry for device "naa.5000000000000001" from Plugin "NMP" failed. Not supported 2016-11-27T10:25:05.885Z cpu0:65593)ScsiDeviceIO: 9167: Get VPD 86 Inquiry for device "naa.5000000000000001" from Plugin "NMP" failed. Not supported 2016-11-27T10:25:05.887Z cpu0:65593)ScsiDeviceIO: 7851: Could not detect setting of QErr for device naa.5000000000000001. Error Not supported. 2016-11-27T10:25:05.891Z cpu0:65593)ScsiDeviceIO: 8365: Could not detect setting of sitpua for device naa.5000000000000001. Error Not supported. 2016-11-27T10:25:05.898Z cpu1:65562)ScsiDeviceIO: 2948: Cmd(0x439500b60000) 0x1a, CmdSN 0x238 from world 0 to dev "naa.5000000000000001" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0. 2016-11-27T10:25:05.905Z cpu0:65593)LVM: 11136: Device naa.5000000000000001:1 detected to be a snapshot: 2016-11-27T10:25:05.905Z cpu0:65593)LVM: 11143: queried disk ID: <type 2, len 14, lun 0, devType 0, scsi 0, h(id) 4847995147196820166> 2016-11-27T10:25:05.905Z cpu0:65593)LVM: 11150: on-disk disk ID: <type 0, len 11, lun 0, devType 0, scsi 0, h(id) 8386732072863042940> 2016-11-27T10:25:05.914Z cpu0:65593)ScsiEvents: 300: EventSubsystem: Device Events, Event Mask: 40, Parameter: 0x43029e372700, Registered! 2016-11-27T10:25:05.914Z cpu0:65593)ScsiEvents: 300: EventSubsystem: Device Events, Event Mask: 200, Parameter: 0x43029e372700, Registered! 2016-11-27T10:25:05.914Z cpu0:65593)ScsiDevice: 4265: Successfully registered device "naa.5000000000000001" from plugin "NMP" of type 0
The datastore obviously gets detected as snapshot. I don't know why and I've only seen this in ESXi 6.5 but it's very easy to fix. Get a list of VMFS snapshots with ESXCLI:
# esxcli storage vmfs snapshot list 583b1a72-ade01532-55f6-f44d30649051 Volume Name: usbflash VMFS UUID: 583b1a72-ade01532-55f6-f44d30649051 Can mount: true Reason for un-mountability: Can resignature: true Reason for non-resignaturability: Unresolved Extent Count: 1
Use the VMFS UUID to mount the datastore:
# esxcli storage vmfs snapshot mount -u 583b1a72-ade01532-55f6-f44d30649051
The datastore should now show up within /vmfs/volumes and the Embedded Host Client.
Create VMFS5/6 Datastore on USB drives
Creating VMFS5 or VMFS6 datastores works just as in previous versions:
- Connect to the ESXi host with SSH
- Stop the USB arbitrator service. This service is used to passthrough USB device from an ESX/ESXi host to a virtual machine. (When disabling it, you can no longer passthrough USB devices to VMs)
~ # /etc/init.d/usbarbitrator stop
- (optional) Use this command to permanently disable the USB arbitrator service after reboot.
~ # chkconfig usbarbitrator off
- Plug in the USB device to your ESXi host. While connecting the USB device you can either watch /var/log/vmkernel.log to identify the device name or identify it within /dev/disks.
Devices are either named mpx.vmhbaXX or with an naa.X number.~ # ls /dev/disks/
- Write a GPT label to the device (Assuming that the Device ID is naa.5000000000000001)
~ # partedUtil mklabel /dev/disks/naa.5000000000000001 gpt
- To create a partition you need to know the start sector, end sector, which depends on the device size and the GUID.
The start sector is always 2048
The GUID for VMFS is AA31E02A400F11DB9590000C2911D1B8
The end sector can be calculated with the following formula (Use the numbers from getptbl):~ # partedUtil getptbl /dev/disks/naa.5000000000000001 gpt 15566 255 63 250069680
15566 * 255 * 63 - 1 = 250067789
You can also calculate the end sector with the following command:
~ # eval expr $(partedUtil getptbl /dev/disks/naa.5000000000000001 | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1 250067789
- Create the VMFS partition (Replace with your end sector)
~ # partedUtil setptbl /dev/disks/naa.5000000000000001 gpt "1 2048 250067789 AA31E02A400F11DB9590000C2911D1B8 0"
- Format the partition with VMFS5 or VMFS6
# VMFS v5 ~ # vmkfstools -C vmfs5 -S USB-Datastore /dev/disks/naa.5000000000000001:1 # VMFS v6 ~ # vmkfstools -C vmfs6 -S USB-Datastore /dev/disks/naa.5000000000000001:1
The USB-Datastore should now appear in your datastores view.
This is how your command output should look like:
[root@esx4:~] partedUtil mklabel /dev/disks/naa.5000000000000001 gpt [root@esx4:~] eval expr $(partedUtil getptbl /dev/disks/naa.5000000000000001 | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1 250067789 [root@esx4:~] partedUtil setptbl /dev/disks/naa.5000000000000001 gpt "1 2048 250067789 AA31E02A400F11DB9590000C2911D1B8 0" gpt 0 0 0 0 1 2048 250067789 AA31E02A400F11DB9590000C2911D1B8 0 [root@esx4:~] vmkfstools -C vmfs6 -S USB-Datastore /dev/disks/naa.5000000000000001:1 create fs deviceName:'/dev/disks/naa.5000000000000001:1', fsShortName:'vmfs6', fsName:'USB-Datastore' deviceFullPath:/dev/disks/naa.5000000000000001:1 deviceFile:naa.5000000000000001:1 ATS on device /dev/disks/naa.5000000000000001:1: not supported . Checking if remote hosts are using this device as a valid file system. This may take a few seconds... Scanning for VMFS-3/VMFS-5 host activity (512 bytes/HB, 2048 HBs). Creating vmfs6 file system on "naa.5000000000000001:1" with blockSize 1048576, unmapGranularity 1048576, unmapPriority default and volume label "USB-Datastore". Successfully created new volume: 583b0efb-33b7ba1e-57e7-001fc69b9659
USB Datastore on USB drives with >2TB
Large drives still do not work in ESXi 6.5:
2016-11-27T10:18:38.213Z cpu4:66004)WARNING: ScsiPath: 7481: Adapter Invalid does not exist 2016-11-27T10:18:38.216Z cpu4:65939)WARNING: ScsiPath: 7481: Adapter Invalid does not exist 2016-11-27T10:18:38.216Z cpu1:65942)VMK_PCI: 393: Device 0000:00:14.0 name: vmhba32 2016-11-27T10:18:38.216Z cpu1:65942)DMA: 646: DMA Engine 'vmhba35' created using mapper 'DMANull'. 2016-11-27T10:18:38.230Z cpu2:65593)ScsiScan: 446: Path 'vmhba35:C0:T0:L0': Failed to read VPD Serial id page: Not supported 2016-11-27T10:18:38.233Z cpu2:65593)ScsiScan: 446: Path 'vmhba35:C0:T0:L1': Failed to read VPD Serial id page: Not supported 2016-11-27T10:18:38.234Z cpu2:65593)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 1: No connection 2016-11-27T10:18:38.237Z cpu7:65593)ScsiScan: 446: Path 'vmhba35:C0:T0:L1': Failed to read VPD Serial id page: Not supported 2016-11-27T10:18:38.238Z cpu7:65593)StorageApdHandler: 977: APD Handle Created with lock[StorageApd-0x43029e35d910] 2016-11-27T10:18:38.238Z cpu7:65593)ScsiEvents: 501: Event Subsystem: Device Events, Created! 2016-11-27T10:18:38.239Z cpu7:65593)VMWARE SCSI Id: Could not get disk id for vmhba35:C0:T0:L1 2016-11-27T10:18:38.239Z cpu7:65593)ScsiDeviceIO: 9120: Get VPD 80 Inquiry for device "t10.WD______SES_Device______WMC1T2972871____" from Plugin "NMP" failed. Not supported 2016-11-27T10:18:38.239Z cpu7:65593)ScsiDeviceIO: 9167: Get VPD 86 Inquiry for device "t10.WD______SES_Device______WMC1T2972871____" from Plugin "NMP" failed. Not supported 2016-11-27T10:18:38.242Z cpu7:65593)ScsiDeviceIO: 7851: Could not detect setting of QErr for device t10.WD______SES_Device______WMC1T2972871____. Error Not supported. 2016-11-27T10:18:38.242Z cpu7:65593)ScsiEvents: 300: EventSubsystem: Device Events, Event Mask: 40, Parameter: 0x43029e383880, Registered! 2016-11-27T10:18:38.242Z cpu7:65593)ScsiEvents: 300: EventSubsystem: Device Events, Event Mask: 200, Parameter: 0x43029e383880, Registered! 2016-11-27T10:18:38.242Z cpu7:65593)ScsiDevice: 4265: Successfully registered device "t10.WD______SES_Device______WMC1T2972871____" from plugin "NMP" of type 13 2016-11-27T10:18:38.244Z cpu7:65593)ScsiScan: 446: Path 'vmhba35:C0:T0:L0': Failed to read VPD Serial id page: Not supported 2016-11-27T10:18:38.245Z cpu7:65593)StorageApdHandler: 977: APD Handle Created with lock[StorageApd-0x43029e213fd0] 2016-11-27T10:18:38.245Z cpu7:65593)ScsiEvents: 501: Event Subsystem: Device Events, Created! 2016-11-27T10:18:38.246Z cpu7:65593)VMWARE SCSI Id: Could not get disk id for vmhba35:C0:T0:L0 2016-11-27T10:18:38.246Z cpu7:65593)ScsiDeviceIO: 9120: Get VPD 80 Inquiry for device "t10.WD______My_Book_1140____WMC1T2972871____" from Plugin "NMP" failed. Not supported 2016-11-27T10:18:38.246Z cpu7:65593)ScsiDeviceIO: 9167: Get VPD 86 Inquiry for device "t10.WD______My_Book_1140____WMC1T2972871____" from Plugin "NMP" failed. Not supported 2016-11-27T10:18:38.248Z cpu7:65593)ScsiDeviceIO: 7851: Could not detect setting of QErr for device t10.WD______My_Book_1140____WMC1T2972871____. Error Not supported. 2016-11-27T10:18:38.251Z cpu2:65563)ScsiDeviceIO: 2948: Cmd(0x439500b1fe80) 0x1a, CmdSN 0x6ce from world 0 to dev "t10.WD______My_Book_1140____WMC1T2972871____" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0. 2016-11-27T10:18:38.253Z cpu7:65593)ScsiDeviceIO: 8365: Could not detect setting of sitpua for device t10.WD______My_Book_1140____WMC1T2972871____. Error Not supported. 2016-11-27T10:18:38.253Z cpu7:65593)ScsiDevice: 3163: Failing registration of device 't10.WD______My_Book_1140____WMC1T2972871____': Unsupported sector size:4096 2016-11-27T10:18:38.253Z cpu7:65593)ScsiEvents: 545: Event Subsystem: Device Events, Destroyed! 2016-11-27T10:18:38.253Z cpu7:65593)WARNING: NMP: nmp_RegisterDevice:851: Registration of NMP device with primary uid 't10.WD______My_Book_1140____WMC1T2972871____' failed. Not supported 2016-11-27T10:18:38.253Z cpu7:65593)StorageApdHandler: 1062: Freeing APD handle 0x43029e213fd0 [] 2016-11-27T10:18:38.253Z cpu7:65593)StorageApdHandler: 1147: APD Handle freed!
thanks for the information, now I definitely know that my 3TB WD Elements 25A won't work :/
by chance do you know the reason why they limited the possible usb size to 2TB?
Thanks for this great information. It worked for me perfectly last year with 6.0 but now I'm having trouble with 6.5. I have rebuilt my test environment from scratch on 6.5 but I really need this USB datastore piece to work to complete it so that ESXi has a place to write the system logs and HA info.
First of all, I'm not sure it's listing the disks correctly because my USB device starts with a "t10" instead of a "naa" or "mpx". So, I go through the process with the "t10" listing and everything goes fine until the end where it fails to create the file system. Can you give me any ideas about how to fix this?
Here is the output:
[root@VSANTEST2:~] /etc/init.d/usbarbitrator stop
UsbUtil: Releasing all USB adapters to VMkernel
watchdog-usbarbitrator: Terminating watchdog process with PID 66555
usbarbitrator stopped
[root@VSANTEST2:~] chkconfig usbarbitrator off
[root@VSANTEST2:~] partedUtil mklabel /dev/disks/t10.SanDisk00Ultra_Fit000000000000004C530001070414108122 gpt
[root@VSANTEST2:~] ls /dev/disks
naa.2020030102060804 vml.0000000000766d68626133323a303a30
naa.2020030102060804:1 vml.0000000000766d68626133323a303a30:1
naa.2020030102060804:5 vml.0000000000766d68626133323a303a30:5
naa.2020030102060804:6 vml.0000000000766d68626133323a303a30:6
naa.2020030102060804:7 vml.0000000000766d68626133323a303a30:7
naa.2020030102060804:8 vml.0000000000766d68626133323a303a30:8
naa.2020030102060804:9 vml.0000000000766d68626133323a303a30:9
t10.ATA_____Samsung_SSD_850_EVO_250GB_______________S2R5NXAH385356T_____ vml.0000000000766d68626133343a303a30
t10.ATA_____Samsung_SSD_850_EVO_M.2_250GB___________S24BNXAH303739K_____ vml.0100000000533234424e5841483330333733394b202020202053616d73756e
t10.SanDisk00Ultra_Fit000000000000004C530001070414108122 vml.0100000000533252354e58414833383533353654202020202053616d73756e
[root@VSANTEST2:~] partedUtil getptbl /dev/disks/t10.SanDisk00Ultra_Fit000000000000004C530001070414108122
gpt
1890 255 63 30375936
[root@VSANTEST2:~] eval expr $(partedUtil getptbl /dev/disks/t10.SanDisk00Ultra_Fit000000000000004C530001070414108122 | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1
30362849
[root@VSANTEST2:~] partedUtil setptbl /dev/disks/t10.SanDisk00Ultra_Fit000000000000004C530001070414108122 gpt "1 2048 30362849 AA31E02A400F11DB9590000C2911D1B8 0"
gpt
0 0 0 0
1 2048 30362849 AA31E02A400F11DB9590000C2911D1B8 0
[root@VSANTEST2:~] vmkfstools -C vmfs6 -S USB-Datastore /dev/disks/t10.SanDisk00Ultra_Fit000000000000004C530001070414108122
create fs deviceName:'/dev/disks/t10.SanDisk00Ultra_Fit000000000000004C530001070414108122', fsShortName:'vmfs6', fsName:'USB-Datastore'
deviceFullPath:/dev/disks/t10.SanDisk00Ultra_Fit000000000000004C530001070414108122 deviceFile:t10.SanDisk00Ultra_Fit000000000000004C530001070414108122
ATS on device /dev/disks/t10.SanDisk00Ultra_Fit000000000000004C530001070414108122: not supported
.
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs6 file system on "t10.SanDisk00Ultra_Fit000000000000004C530001070414108122" with blockSize 1048576, unmapGranularity 1048576, unmapPriority default and volume label "USB-Datastore".
Failed to create VMFS on device t10.SanDisk00Ultra_Fit000000000000004C530001070414108122
/dev/disks/t10.SanDisk00Ultra_Fit000000000000004C530001070414108122: Permission denied. (Have you set the partition type to 0xfb?)
Error: Permission denied
[root@VSANTEST2:~]
Same issue no idea how to get it to work.
Hello you need to add at the end of the device name :1 for example (vmkfstools -C vmfs5 -S LOCAL-USB /dev/disks/mpx.vmhba32
:C0:T0:L0:1) and not working with vmkfstools -C vmfs5 -S LOCAL-USB /dev/disks/mpx.vmhba32
:C0:T0:L0 (":1")
Chris, here's what got it to work for me. The clue is in this output that you pasted above "(Have you set the partition type to 0xfb?)"
1. Insert USB drive into Windows PC
2. Launch Paragon Hard Disk Manager 15 Premium (or similar program that can do this function)
3. Right click the USB, click Change Partition ID
4. Enter FB for the Partition ID.
5. Apply changes, eject USB, insert into ESXi machine, do again. It will work this time. At least it did for me, and I had the exact error you got.
after doing this will it then auto-mount or will you need to manually mount those datastores every time when the host reboots?
Use the VMFS UUID to mount the datastore:
Pingback: USB Devices as VMFS Datastore in vSphere ESXi 6.0 | Virten.net
I am running W10 VMPlayer and ESXi 6.5
Following your instructions
From the GUI enable SSH
/etc/init.d/usbarbitrator stop
UsbUtil: Releasing all USB adapters to VMkernel
watchdog-usbarbitrator: Terminating watchdog process with PID 66553
usbarbitrator stopped
ls -al /dev/disks/
.. Text removed
lrwxrwxrwx 1 root root 48 Jan 7 20:25 vml.0000000000766d68626133323a303a30 -> t10.SanDisk_Cruzer_Blade____4C530001090529103131
lrwxrwxrwx 1 root root 50 Jan 7 20:25 vml.0000000000766d68626133323a303a30:1 -> t10.SanDisk_Cruzer_Blade____4C530001090529103131:1
partedUtil mklabel /dev/disks/t10.SanDisk_Cruzer_Blade____4C530001090529103131 gpt
partedUtil getptbl /dev/disks/t10.SanDisk_Cruzer_Blade____4C530001090529103131
gpt
947 255 63 15223808
partedUtil setptbl /dev/disks/t10.SanDisk_Cruzer_Blade____4C530001090529103131 gpt "1 2048 15213554 AA31E02A400F11DB9590000C2911D1B8 0"
gpt
0 0 0 0
1 2048 15213554 AA31E02A400F11DB9590000C2911D1B8 0
vmkfstools -C vmfs6 -S USB-Datastore /dev/disks/t10.SanDisk_Cruzer_Blade____4C530001090529103131:1
create fs deviceName:'/dev/disks/t10.SanDisk_Cruzer_Blade____4C530001090529103131:1', fsShortName:'vmfs6', fsName:'USB-Datastore'
deviceFullPath:/dev/disks/t10.SanDisk_Cruzer_Blade____4C530001090529103131:1 deviceFile:t10.SanDisk_Cruzer_Blade____4C530001090529103131:1
ATS on device /dev/disks/t10.SanDisk_Cruzer_Blade____4C530001090529103131:1: not supported
.
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs6 file system on "t10.SanDisk_Cruzer_Blade____4C530001090529103131:1" with blockSize 1048576, unmapGranularity 1048576, unmapPriority default and volume label "USB-Datastore".
Successfully created new volume: 5871519d-0e791556-7a97-000c29da1612
And it appears in the GUI screen
I will follow your step to set USB disk but have the error message at last.
can you help to solve this problem thanks a lot.
[root@localhost:~] vmkfstools -C vmfs5 -S USB-Datastore /dev/disks/naa.5000000000000001
create fs deviceName:'/dev/disks/naa.5000000000000001', fsShortName:'vmfs5', fsName:'USB-Datastore'
deviceFullPath:/dev/disks/naa.5000000000000001 deviceFile:naa.5000000000000001
ATS on device /dev/disks/naa.5000000000000001: not supported
.
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs5 file system on "naa.5000000000000001" with blockSize 1048576 and volume label "USB-Datastore".
Failed to create VMFS on device naa.5000000000000001
/dev/disks/naa.5000000000000001: Permission denied. (Have you set the partition type to 0xfb?)
Error: Permission denied
(Have you set the partition type to 0xfb?)
The command should be "vmkfstools -C vmfs5 -S USB-Datastore /dev/disks/naa.5000000000000001:1".
ends with ":1".
Hi Ernest
You are using /dev/disks/naa.5000000000000001
but I used /dev/disks/t10.SanDisk_Cruzer_Blade____4C530001090529103131:1
What name is your USB actually called from
ls -al /dev/disks/
Regards Conwyn
Hi,
du u know what is the maximum size for USB3.0 HDD on esx6 for mounting as VMFS Datastorage.
In my case i need to mount 6 TB in a single Disk over USB.
As far as I know there are problems with disks larger than 2TB.
Hello,
I was able to mount a Western Digital 5tb drive to an ESXi 6.5 host.
then used the above commands to create a datastore on the usb drive.
Then when I unmounted it, o move it to a different ESXi 6.5 host, it would not mount and it displays "t10.WD______easystore_2624__WX11E69AAAXH____",
Any suggerstions?
Is there any way I can create such a datastore on the existing ESXi USB boot drive? There's 25Gb of unused space there, it would be great if I could use that for VMs instead of having to plug in a second USB drive. I can't just plug in an SSD for this either, I would need to buy another HBA for more SAS ports :(
Not out of the box but you can try this: http://www.horizonbits.com/2017/02/19/squeezing-esxi-on-usb/
Something to add to this (albeit not directly related): I upgraded to ESXi 6.5 and had previously used a JMicron external USB to SATA enclosure. Post upgrade to 6.5 I could only use one JMicron enclosure at a time.
Using ls /dev/disks they were using identical identifies. In vCenter under the host storage, I could see that it listed one "path" as down and the other "path" as active although they were previously two separate devices in ESXi 6.0.
In the end I had to ditch the 1TB external disk and only use the 2TB disk.
Not what I wanted to do but with the new storage driver things changed.
Thanks! I have just successfully added my 1T external HD to my ESXi. This was my very 1st attempt on doing so. This is an awesome tutorial.
Thanks for great article. I got the following question:
I formatted the USB drive on one ESXi and it's showing as datastore and I copied VMDK files.
When I try to insert the USB drive into another ESXi, the USB drive gets sort of recognized (grayed out). When I try to mount, I get error message.
Is it possible if you could show what is the correct way to do this?
Thanks.
I had issue in that step:
vmkfstools -C vmfs5 -S USB-Stick /dev/disks/mpx.vmhba36\:C0\:T0\:L0:1
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs5 file system on "mpx.vmhba32:C0:T0:L0:1" with blockSize 1048576 and volume label "usb-iso".
Usage: vmkfstools -C [vmfs3|vmfs5] /vmfs/devices/disks/vml... or,
vmkfstools -C [vmfs3|vmfs5] /vmfs/devices/disks/naa... or,
vmkfstools -C [vmfs3|vmfs5] /vmfs/devices/disks/mpx.vmhbaA:T:L:P
Error: vmkfstools failed: vmkernel is not loaded or call not implemented.
I used Vsphere 5.0
Tks
I had issue in that step:
vmkfstools -C vmfs5 -S USB-Stick /dev/disks/mpx.vmhba36\:C0\:T0\:L0:1
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs5 file system on "mpx.vmhba32:C0:T0:L0:1" with blockSize 1048576 and volume label "usb-iso".
Usage: vmkfstools -C [vmfs3|vmfs5] /vmfs/devices/disks/vml... or,
vmkfstools -C [vmfs3|vmfs5] /vmfs/devices/disks/naa... or,
vmkfstools -C [vmfs3|vmfs5] /vmfs/devices/disks/mpx.vmhbaA:T:L:P
Error: vmkfstools failed: vmkernel is not loaded or call not implemented.
I used Vsphere 5.0
Tks
Hello! I have two host esxi6.5. If I connect one usb disk to a usb hub and then i connect one usb port to the first host and other usb hub port to the secondary host esxi. Can I have one shared storage? Can I configure a cluster for High availability or Fault tolerance? Thanks, MGB
No, that does not work. You can't connect a USB disk to multiple hosts. USB doesn't support that.
Does anyone know if storing the System logs only on a USB device will cause issues? We have 4 R740 Hosts using all flash VSAN and no extra local disks for a datastore. ESXi is installed on the IDSDM module flash cards and for some reason the logs don't like to live on that datastore. We currently get the warning that System logs are stored on -non-persistent storage. Is it a better design to use the VSAN datastore for the logs rather than USB?
Forgot to mention we are using the latest version of ESXi 6.5 Build 6765664 and were going to use a 64GB USB flash for log storage. I ran across this article because ESXI would not detect or flash drives as expected. Did not know Vmware did not support USB drives for datastores (even if it for system logs only??)
Syslog.global.logDir is set to /stratch/log
I found this Vmware article
https://kb.vmware.com/s/article/2147541
that states logs on VSAN datastores can cause issues. So we are back to USB or Syslog or live with ramdisk warning. Any advice? Leaning towards USB but also read this article stating USB drives are not a great solution either.
https://cormachogan.com/2015/02/24/vsan-considerations-when-booting-from-usbsd/
UPDATE: We decided to try the USB route and the instructions worked with one small exception. We had to turn off the USB Arbitrator Service not just temporarily disable it. We tried to keep the service on but when it turned on our USB datastore became inaccessible.
Thanks for the information, that was helpful.
hi everybody,
this method worked for me too without any problem. but i dont know why i cant see my usb storage and when power on/off my harddisk nothing change problem continues. but i noticed that vml identifier name change it was named before mpx.vmhba38:C0:T0:L0 bu not i see it as mpx.vmhba40:C0:T0:L0 and when i plug usb in and out everytime this changes and still can reach my vmdk files. how can i recover that issue ?
Thanks for this well crafted article. Question for everyone ... How do I make it stick? if I bounce the esxi server the USB datastore does not auto moint
Hi. Great article. I was just wondering if you have to leave the USB arbitrator service off in order for this to work or can you re-enable it after it is set up. The reason I ask is that I'd like to add an external USB drive as a datastore but also pass a different USB port through to a VM.
while i create VMFS partition using below command i getting the "read only file" error message, can anyone help?
[root@localhost:~] partedUtil setptbl /dev/disks/naa.5000000000000001 gpt "1 2048 1953520064 AA31E02A400F11DB9590000C2911D1B8 0"
gpt
0 0 0 0
1 2048 1953520064 AA31E02A400F11DB9590000C2911D1B8 0
Error: Read-only file system during write on /dev/disks/naa.5000000000000001
SetPtableGpt: Unable to commit to disk
I just wanted to say thank you for this. It isn't perfect, but it *does* work. Thanks for the effort!
Are this working in 6.7 also ?
Awesome howto!!!! Many thanks - works a treat :-)
Hi All. I used this tutorial to mount my 1TB USB HD. I backed up the VMs.
I installed the ESXI 6.0 the I followed this tutorial again to access the USB HD to recover the VMs. But I got a bad surprise. The disc was empty.
I can't open the HD in Windows or Linux.
This tutorial is just for USB HD without data?
When we create a partition with the command partedUtil we lost all data?
I didn't format the HD.
Thank you SOOOO much! in the sea of ESXi info, this post saved my life.
I recently have to switch server - not planned, but my old RAID can see HBA drive, and the new one HP P410i can only presents the "logical" drive. That made my life like hell! Your post let me reunited with my most valuable VM that got locked up in a single drive.
Thanks you!!! x 1000
We were using 6.0.0 with multiple external USB drives on Mac Pro hardware (OS X Vms) due to how these machines are setup (limited internal disk). Worked fine. Upgrade failed to mount any of the drives, and now it can only see 1 at a time. This is a no-go for us, but 6.7 is REQUIRED to run Mojave. If no solution is found, we will be ditching VMWare for hosted providers entirely.
We upgraded 2 of our Mac Pro 6.1 machines (the cylinder ones) from ESXi 6.0.0u2, where we had 3 external 1 and 2 TB usbc drives (all samsung, same models), to 6.7, and now only 1 drive is available as a datastore on each. The ESXi kernel definitely sees all the drives, /dev/char/vmkdriver/usbdevices shows all 3 with serial numbers, bus assignments and everything, but it only maps the first one to a disk device. Evaluating thunderbolt drives now, since internal pcie drive upgrades for these macs are ridiculously expensive. Note that VMWare does not support VMFS/Datastores on USB, and will tell you such if you open a support request, even if it was working on earlier versions (was told this by a support engineer).
This worked like a charm :-D. You rock!
Think this can be done with a SD card as a datastore?
This was super helpful!. Thank you very much for the clear and well laid info.
Kudos!!!
This worked great under ESXi 6.7, thank you!
I am not able to mark the device as Flash though. I am using a USB3 adapter and port and the device is SSD.
I have an obscure need for the drive to be marked as flash (not VSAN) and was only trying this out so I could use the USB flash drive I have temporarily.
Do you know any tricks to get the USB drive to mark as flash or is that related to the controller?
Thank you so much, it helped a lot. You are a God.
Thank you for this guide. I've recently applied this on an ESXi 6.7 U2 machine and it works great. However, doing some speed tests I noticed my max benchmark is reaching about 265MB/s with an SSD connected to a SATA to USB3 adapter that should be able to do better. Any suggestions?
Thank you very much for this guide, it works like a charm using an Intel NUC placing the Datastore on a Samsung T5.
But I do have a problem and question...
Part of the requirements for this project is once the Datastore is built with the VM's to create a mirror Image/Clone of the T5 on another T5. Unfortunately the clone T5 will not work...
The clone T5 is seen with ls /dev/disks/ and in the GUI the T5 is listed under "Devices" tab of Storage but there is nothing under the "Datastores" tab.
So how can I fix this so there can be clone images of the T5?? Thanks
I think it might be recognized as a Snapshot Volume. Check with:
# esxcli storage vmfs snapshot list
If the volume is listed you either have to resignature or directly mount it. If the clone is permanent, resignature is the better option.
esxcli storage vmfs snapshot --help ...will get you through the remount process.
Eg.:
esxcli storage vmfs snapshot resignature -l datastorename
That's the first thing I checked... It is not a Snapshot Volume, when I execute # esxcli storage vmfs list nothing returns...
- Although as I said # ls /dev/disks/ does show the clone...
- And # fdisk -l also shows the clone...
- Both the GUID and UUID's are identical between the original and the clone...
- Looking at the vmkernal.log there are no errors or warnings now but yesterday when I first connected the clone T5 there was
"NFS 1227: Invalid volume UUID naa.5000000000000001:1"
and
"FSS 6092: No FSS driver claimed device 'naa.5000000000000001:1': No filesystem on the device"
The issue is resolved…
I ran # partedUtil getptbl “/dev/disks/naa.500000000000000” and part of the returned information was a Partition GUID of EBD0A0A2B9E5443387C068B6B72699C7 which is for Basic Data NOT VMFS Datastore… While the original T5 returned a Partition GUID of AA31E02A400F11DB9590000C2911D1B8 which is correct…
The particular Clone/Image device I have is a Logicube Zclone Xi and an option for Imaging is a “Clever Image” which only writes used space… When I reran the Image as a “Mirror Image” the Clone T5 now has an operational Datastore and the Partition GUID is correct…
The obscure stuff we learn…..
Hi. I've successfully mounted my usb as usb datastore. Now If I want to backup few files and then want to connect to some other host and restore those files in new host, is it possible?
Hello Azhar, have you found a solution to do this?
I need some feed back.i have mounted a USB as a VMFS data store the procedure worked fine. Every thing worked as intended.
I can see USB data store but the problem is that i want to use this as a backup so i case of emergency i need this VMFS volume to be available on an other host.
How can unmount this data store and mount it to another data store on different esxi host without destroying the back up data ?? any possible solution
Great tutorial. Worked like a charm.
Great article! Before, I used the methods you discuss here. Now, my PowerShell script, which was created based on your post, does everything. Thanks a lot! Here’s the link, hope it will be useful: https://www.vmwareblog.org/vmfs-datastore-usb-drive/
Thank you! Worked great!
This worked for me on ESXi 6.7 using a WD MyBook 3TB USB3 HDD.
Excelente aporte!
tenía el mismo problema con ESXI 6.7. seguí los pasos y agregué un HDD vía USB 3.0 y funcionó correctamente. hay que tener en cuenta que el usbarbitrator se debe apagar permanentemente, si no al reiniciar el ESX se perderá nuevamente el nuevo datastorage.
Gracias!
Thank you very much! Used this blog to add a 1 TB Samsung EVO 860 as an external USB datastore via USB 3.0 to my NUC PC running ESXi 6.7U3. Worked like a charm. I was very surprised that the additional hard disk was not recognized right away automatically.
With respect to the USB arbitrator part. I stopped and deactivated it according to your howto. Does this now mean I can no longer "mount" USB devices connected to the NUC to any VM running on it?
Thanks so much for this post!
I just upgraded from ESXi 6.0 to the latest 6.7, and everything went well except that I couldn't see my datastores from USB storage, though I could see the drives themselves.
Doing:
# esxcli storage vmfs snapshot list
and then
# esxcli storage vmfs snapshot mount -u [uuid from previous step]
worked like a charm!!
Thanks for figuring out how to mount a USB as a datastore. Worked like a charm!
I'm moving an Esxi lab across the country on a very low budget. The target server/datacenter does not support e-sata, so I loaded up an external hard drive via e-sata, shipped it, and am now unloading it via USB. It seems like my situation should be common enough for VMWare to expect it, but I guess we all think that about our specific problems. Anyway thanks for sherlocking the snapshot misdirection. We are uploading now!
Thank you, it worked perfectly for my ESXi 6.7.0 Update 3 (Build 15160138) running on Intel NUCi5 and using a Samsung 860 EVO via JMicron USB2SATA adapter.
However: How can I continue using USB devices in hosts now that the usbarbitrator service is turned off? Can I selectively mask/blacklist the JMicron from getting arbitrated and all the rest being available for mounting in a VM? I want to mount a smartcard reader connected on the NUC/ESXi host into a vm
To add to my last comment: I've solved the challenge:
https://communities.vmware.com/message/2935267#2935267
MarkusD - thank you for your guide, it worked as described!
Are you aware of a method to allow a reconnected device to be automatically re-enabled for passthrough and reconnect to the VM, without each time re-entering the command:
localcli hardware usb passthrough device enable --device [Bus:Dev:VendorID:ProductID]
and re-adding the device in ESXi?
THANK YOU THANK YOU THANK YOU!!!!!!!!!!!!!!!
Automounting - Does anyone have any tips/tricks for enabling this for automounting is the entire esxi server reboots (gracefully or not) ?
Hello,
Does anyone a suggestion on how to move the USB drive, that is already formatted to another ESXi host? The USB drive is recognized and seen in the devices but will not allow the mounting of it.
Hello,
I just replied to a message in this thread about this.
I used these commands on a Western Digital usb drive that was 5tb in size.
When I un-mounted it, to put on another esxi host, I could not.
The device is seen but cannot access the datastre on it.
Did you find any other info?
Another option is to take the iscsI route with a NAS Drive. Ive done back ups of VMs into the NAS Drive in one network with a ESXi host, then moved it to another network and different ESXi host and opened it up fine.
I'm using esxi 7.0 on an Intel NUC 10. Adding an ADATA 1TB ssd through USB works just fine, however I can't mark it as a flash device in vcenter.
cannot mark $path as "Flash". unable to reconfigure the disk claim rules. the disk is in use"
The disk is _NOT_ in use. I've zeroed it with dd, recreated the partition table to GPT and MSDOS.
I still can't mark the drive as flash to use it in an all-flash vSAN :(
Anyone have any ideas?
Had the same problem - virten has written another article with the solution...
https://www.virten.net/2020/09/mark-usb-storage-devices-as-flash-fails-with-the-disk-is-in-use-error-in-esxi/
Question, when I mount the drive I cannot move the flat vmdk file to any other datastore on this host. The FS on the USB is VMFS5 and the other datastores are VMFS6. I cannot even copy the file to my workstation with the datastore browser. Is the file locked? It was a second hard drive for a guest OS across two datastores...primary vmdk on one this vmdk on the other.
How do I unmount this from usb drive once the data is moved?
Thx you very much.
You save my day.
Perfect explaintion and it still works (for me :)
Thank you so much! Brilliant!
This article saved my bacon! Thanks!
Worked perfectly! Thanks for the article. Now I have a USB drive to keep all the ISOs and stuff on that were taking up too much space on my NUCs internal SSD.