Building an ESX3i White Box: Enterprise Virtualization for Less

Server virtualization is and has been a hot topic in the IT field in recent years. The use of virtualized servers allows businesses to expand their infrastructure in density (vertically) instead of in physical machine quantity (horizontally). Among other benefits this minimizes the required floor space, power consumption, and physical hardware required to keep the business running. These particular benefits are very appealing to large corporations which own and operate their own datacenters, and to the individual or small business owner who is renting rack space in a secure datacenter the savings created by server virtulization can be even more importantant. This article documents the step required to build a relatively low cost server virtualization platform with most of the features being used by billion dollar corporations today.
If you wish to try ESXi before you build a server specifically for it, simply follow the Software section of this article and plug the loaded USB drive into an available computer which can boot from a USB slot. Most computers made within the past few years should work to some degree, depending on the hardware and if ESXi comes with drivers for it.

Software
Although there are various vendors currently offering virtualization software, the leading choice among top businesses currently is VMware. VMware offers a full range of virtualization products, but for this article we will focus on their most recent release; ESXi. This software is a minimal version of their popular ESX Server product line which provides “an enterprise-class hypervisor with a thin 32 MB footprint”. It is intended to be installed on an SD card which is included with a select number of enterprise class servers from hardware vendors such as Dell, IBM, and HP. To learn more about ESXi, click the following link http://vmware.com/products/esxi/.

Edit: VMware now gives users a license of ESXi for free after registration!  You are no longer limited to a trial.
Although the licensing for ESXi is currently $495, VMware allows anyone to download a 60 day trial of the product for free. This evaluation copy was used throughout the system build, and can be upgraded to the retail copy at the end of the trial period if desired.

Note: The following instructions are a summary of the instructions found at http://vmetc.com/2008/02/05/create-your-own-bootable-esx-3i-usb-stick/ .
Since USB jump drives are much more common than PCI to SD card adaptors, we will install the software onto a >=1GB USB drive. To start, download an evaluation copy of the software (VMware ESXi 3.5 Installable Update 1) from VMware’s site http://vmware.com/download/vi/ . Next you will need to download a trial copy of WinImage (http://www.winimage.com) and a freeware arc hiver such as 7zip (http://www.7-zip.org). Then perform the following steps:

1. Extract INSTALL.TGZ from the root directory of the ESXi ISO image using 7zip.
2. Extract /usr/lib/vmware/installer/VMware-VMvisor-big-3.5.0-67921.i386.dd.bz2 from INSTALL.TGZ using 7zip
3. Extract VMware-VMvisor-big-3.5.0-67921.i386.dd from VMware-VMvisor-big-3.5.0-67921.i386.dd.bz2 using 7zip
4. Attach the USB flash drive and make sure you no longer need the data on it
5. Use WinImage to transfer VMware-VMvisor-big-3.5.0-67921.i386.dd to the USB flash drive
1. Disk->Restore Virtual Hard Disk image on physical drive…
2. Select the USB flash drive (Warning: If you select the wrong disk you will lose data!)
3. Select the image file VMware-VMvisor-big-3.5.0-67921.i386.dd
4. Confirm the warning message
5. Wait for the transfer to complete
6. Unplug the USB flash drive (Warning: If you forget to unplug the flash drive from the PC you might lose the data on your hard drives the next time you boot!)

Hardware
Now that we have our bootable ESX drive, it is time to build the physical box which will host the virtual machines. Because ESX is meant to be an enterprise product, it is only sold and supported on a select number of expensive server platforms which sell for anywhere from $3000-$30,000. However even though these are the only officially supported platforms VMware lists, the ESXi hypervisor can actually work with a variety of undocumented hardware. Because of this, building a white box often involves finding out by trial and error which hardware works and which doesn’t. The following, guaranteed to work, configuration was used during my build to fit my personal needs and desired budget, but other more or less powerful configurations may also work if suitable. All of the parts used were purchased from online PC hardware providers and the prices listed may reflect currently unavailable sales or rebates.

Part/ Price
4U rack mountable case: Norco RPC-800 / $74
8GB Corsair XMS2 DDR2 800 RAM (4×2GB) / $152
Corsair CMPSU-550VX 550W Power Supply / $80
Intel Q9300 Processor / $275
ABIT IP35 Pro Motherboard / $130
Intel EXPI9300PT Gigabit PCI-E Network Card (x2) / $90
2GB Sony USB Micro Drive / $13
Cheap PCI or PCI-E video card (only needed for setup) / $15
1TB HDD (Western Digital Caviar SE16 WD5000AAKS SATA drive x2) / $180
Total $1,009

The hardware used provides a fairly beefy machine (quad core with 8GB RAM and 1TB of disk space) easily capable of running numerous virtual servers simultaneously. For example the above hardware is currently running four different VM’s each with a 3 GHz CPU core and 2GB RAM. The particular motherboard used includes two on-board gigabit NIC’s which are unfortunately not supported by ESXi at this time. Therefore, the two PCI-E Intel NIC’s were included to provide both a VM traffic and an ESX management physical Ethernet port. Because ESXi also does not support IDE devices, the hard drives used must be SATA. And although the motherboard includes onboard raid, the chipset used is only supported in IDE mode (meaning no RAID or AHCI support). If raid is desired, a raid PCI or PCI-E card may be used.

Installation
After setting up the physical ESX box, insert the USB drive with the newly loaded hypervisor into an available USB port. Once the box boots up, a setting needs to be configured in the bios for the server to boot from the USB drive. Under Integrated Peripherals->OnChip PCI Device->USB Device Settings make sure the USB Storage Function is set to Enabled. Then make sure the Hard Disk Boot Priority is set to have the USB drive boot first. Then save the settings and let the machine boot into ESX. You are now ready to configure the ESX server as desired.

Final Thoughts
Server virtualization is an increasingly important skill for IT professionals. While the system described in this article is certainly stable enough to used in a business environment, because it uses white box components and is not officially supported by VMware it is more practical for use in a “home” or lab environment. By using the above procedures ESXi gives the novice techie the ability to learn, play, and benefit from the skills required for in-demand positions in the IT field.

Related Topics

• Remote Shell Access in ESXi
• Keep your VM’s in shape with Auto Tool updates
• How-to Repair a Windows Installation