ESXi Colo part 2: The All-In-One storage and VM host

I had to delay this post due to some projects at work- Enjoy!

After a few weeks of waiting on various parts, I finally have everything together for my ESXi colo box. I finished assembling it after the holidays and got it burnt it in. I left the major config for some downtime around the office. Of course the day I got it racked an immediate work related need for a lab environment came up. The last few days I have spent my spare time configuring the storage passthrough and setting up the All-in-One.

If you are not familiar with the concept of a All-in-One ESXi host, I suggest you check out www.napp-it.org. Gea, the creator of Napp-IT has inspired a new line of high performance, compact, All-in-One storage/VM hosts. From his background in education and small budgets, he decided to ditch the traditional SAN "network" and build redundant All-in-One Hypervisor and storage hosts.

The basic requirements/recommendations of a All-in-One are as follows:

Supermicro X8/X9 motherboard supporting VT-D
Fast processors (Think Dual Quad 2.5 min or an e5-26xx hex core)
As much ram as you can afford (32gb+)
LSI (or rebadged IBM) HBA's
(2) SSD's for SAN OS
X of disks for the shared Pool
SSD's for Cache drives if possible
USB Key for Hypervisor

My system fits the bill perfectly.
X8DTE Motherboard
(2)Intel x5650 processors
192GB DDR3 10600 ECC REG
(2) LSI 9211-8i HBA
2 160gb Intel SSD for Open Indiana
4 Crucial M4 256gb SSD cache/log/ect
4 Crucial M4 512gb SSD VM OS drives
2 3TB Seagate 7200rpm Spindle archival drives
16GB HP Thumbdrive for the Hypervisor

The initial setup via Gea's guide is very simple:
1) Load ESXi onto your thumb drive
2) Install OpenIndiana on to a 40gb Datastore on OS SSD1
3) Pass the LSI HBA through to the OI VM
4) Install Napp-It and mirror the OS drives
5) Create a storage pool with the disks passed through the HBA and enable NFS (In my case, RaidZ10)
6) Create a Vswitch in ESXi with a second dedicated NIC for SAN traffic using the vmnet driver
7) Assign static IP's to the new SAN nic in OpenIndiana
8) Add the datastore in ESXi
9) Install vmware tools on OpenIndiana
10) Start benchmarking!

So far I have seen 1100Mb/s Read and 325Mb/s Write. It has been up solid for a month now hosting ~5 test VM's and I can't wait to start loading it down. Next up I will attach a large pool of spindle disks for some Raid Z testing.