FreeNAS on HP Microserver N54L - Shared Storage for your ESXi

The HP Microserver N54L and its predecessors N40L and N36L are widely spread homeservers. I've written about using it as ESXi host in my Homelab about 2 years ago. Today it is still a great system to be used with ESXi but there are better alternatives, especially due to its limited memory and CPU power. That was the reason why I rededicated my N40L to be central storage and file server with FreeNAS. This post describes what you need to use the N40L as resilient storage and how to configure it properly for ESXi usage. I will talk about NFS sync writes or performance issues with NFS vs. iSCSI and how to workaround them.

freenas-on-hp-microserver-n36l-n40l-n54l

Hardware Guide

  • HP Microserver: If you do not already have a Microserver, go for the N54L.
  • Memory: FreeNAS requires 8GB of RAM to run properly: Kingston KVR1333D3E9SK2 or Corsair CT2KIT102464BA1339.
  • Hard Drives: I recommend the WD Red drives (2TB, 3TB or 4TB). The Red series was specially made for NAS usage. You can also use the WD Velociraptor model which is a little bit faster, but much more expensive. The system has 4 slots, but you can add a 5th drive to the cd/dvd slot.
  • NIC: An additional NIC is recommended for performance and redundancy: Intel Gigabit PCI-E Network Adapter EXPI9301CTBLK
  • USB Flash Drive: I recommend to install Freenas to a USB Flash Drive. The recommended boot device size for FreeNAS is 8GB: SanDisk Cruzer Fit 8GB
  • HP MicroServer Remote Access Card: For out-of-band management.

Shopping Guide

1.

Chassis

HP Microserver Chasis


HP Proliant Microserver N54L

 2.

Memory

hp-memory

Kingston KVR1333D3E9SK2

Corsair CT2KIT102464BA1339

 3.

 Hard Drive

wd-red

WD Red NAS Hard Drive

WD Velociraptor Hard Drive

 4.

 NIC

intel-nic

Intel Gigabit NIC EXPI9301CTBLK 

 5.

USB Flash
nuc-usbstick

SanDisk Cruzer Fit 8GB

 6.

 Remote Access

hp-rac

HP Micro Server Remote Access Card Kit

 

Installation & Configuration

Installation
Installation is very simple. You can use the internal USB port for the USB Flash Drive. If you want to use a 5th hard disk, you can fit it into the cd/dvd slot.

  1. Downlad the latest Release of FreeNAS
  2. Boot the installer (Burn it to a CD or mount it with the Remote Access Card)
  3. Select the 8GB flash drive as target
  4. Enter your root password
  5. Remove the CD and reboot

That's it. You FreeNAS will now reboot and come up with DHCP enabled. The configuration can be done with the web interface.
freenas-web-interface

Initial Configuration

  1. Open the Web Interface
  2. Login as root with the password you set during the installation. This will start the initial configuration wizard.
  3. Set your language and exit the wizard at the second step
  4. If you want to set a static IP address you have to add an interface in Network > Interface 

Create a Volume
With 4 disks, you can build a RAID10. This will give you the best performance. If you have 3 or a 5th disk you can create a RaidZ. This will give you the most capacity.

  1. Open Storage > Volumes > Volumes Manager
  2. Enter a Name
  3. Configure your volume layout. RaidZ (3 or 5 Disks required):
    freenas-raidz
    or Raid10 (4 disks required):
    freenas-raid10
  4. Press Add Volume

FreeNAS running on your N54L is now ready to be used. From that point you can create your shares to use it as Fileserver or shared storage for ESXi.

Create a NFS Share for ESXi

  1. Open the Storage Tab
  2. Select the Volume
  3. Press the Create Dataset icon
    freenas-create-dataset
  4. Enter a Dataset Name and press Add Dataset
  5. Navigate to Sharing > UNIX (NFS) and press Add Unix (NFS) Share
    freenas-add-nfs-share-for-esxi
  6. Browse for the Path you want to share (/mnt/[volume]/[dataset])
  7. Press Advanced Mode
  8. Change Maproot User: to root
  9. Press OK
  10. When it asks to enable services, press Yes

You can mount the share on your ESXi host with  Configuration > Hardware > Storage > Add Storage > Network File System
esxi-mount-nfs

Create a iSCSI Share for ESXi

  1. Navigate to Sharing > Block (iSCSI) > Portals and click Add Portal
  2. Enter a Comment and press OK
  3. Navigate to Initiators and click Add Initiator
  4. Press OK
  5. Navigate to Targets and click Add Target
  6. Enter Target Name, Target Alias and set the Port/Initiator Group ID to 1
  7. Navigate to Extends and click Add Extend
  8. An extend is a file that simulates the Block device. You have to set a fixed size. Select a Extent Name (same as Target Name possible), a Path and the Extent Size
    freenas-add-extend
  9. Navigate to Associated Targets and click Add Target / Extent
  10. Map Target and Extent and press OK
  11. Open Services Tab and enable iSCSI

 

iSCSI vs NFS Performance

Without doing any further optimization like jumbo frames or multipathing, my N40L with 5 WD Red 3TB drives in RAIDZ could achieve the following results.

iSCSI
Write: 72 MB/s - 145 IOPS (I/O Analyzer 512k 100%Read 0%Random)
Read: 98 MB/s - 200 IOPS (I/O Analyzer 512k 0%Read 0%Random)

NFS
Write: 5 MB/s - 20 IOPS (I/O Analyzer 512k 100%Read 0%Random)
Read: 52 MB/s - 120 IOPS (I/O Analyzer 512k 0%Read 0%Random)

That's what I expected. But why is NFS performance so bad? ESXi does synchronous writes operations. The host requests an acknowledgement that the data is actually written to the physical disk before it starts another write. A sync write causes the ZFS filesystem to flush it's ZIL (ZFS Intent Log) to the disk. The ZIL is a log of sync writes that ZFS uses to write data more efficiently to the disk.

So, what can you do to speed NFS up?

  1. Write the ZIL on a separtate SSD. This will speed up NFS a little bit. I could achieve around 30 MB/s with a SLOG (Separate Intent Log).
  2. If you do not care about the integrity of your VM, you can disable sync writes. If you do so, be aware that if your storage crashes while data is cached but not written to the disk you will end up with corrupt data.
    Disable Sync Writes:

    # zfs set sync=disabled [POOL]/[DATASET]

    Enable Sync Writes:

  3. # zfs set sync=standard [POOL]/[DATASET]
    

    You can verify the sync status with

  4. # zfs get sync

 

  1. N54L with ESXi 5.5 but which Raid Card - pingback on January 4, 2015 at 10:37 pm
  2. How to Upgrade FreeNAS 9.x to 9.3 | Virten.net - pingback on January 11, 2015 at 5:23 pm
  3. Great write up thank you.
    I actually had my N40L with ESXi, and then had a VM openindiana presenting the disks (raw disk).
    I am just confused, where does ESXi come into this(blush)?
    i.e. It appears you are booting the server into "free nas" from USB?
    Sorry I am missing something here..
    I am about to build an N54L and may change my setup.
    However my server function is not only "NAS", it is also to host VM's.
    Thanks again.

  4. Usefull post, thanks.
    I have similar configuration in a N54L. With a recent freeNAS update the USB flash memory died. I connect another one, but just reinstalling freeNAS also died. So, I don't know if was just coincidence, or what to think about the same issue 2 times in 24 hours in the same USB port?

  5. @HEDEL ... USB port might be faulty

Leave a Comment

NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Trackbacks and Pingbacks: