In the last little while, I’ve begun to work a lot more with VMware ESXi, in an attempt to understand virtualization better. Along with the benefits it brings to larger companies, I began to see some benefits that it would bring to my personal life. At home, I am constantly developing or experimenting with new technologies in an attempt to understand them and build my skills. The problem is that I tend to do a lot of my experimentation on our home server (an old Dell 5150, with lots of extra hard drives). Since we are using this as a media server, things tend to become difficult for those using the media or data when I need to reboot/install something new. This resulted in much of our data being scattered among our other machines (two MacBook Pro’s and an iMac). All in all, a very messy setup.
Enter ESXi. Virtualized machines brings me the ability to leave a media server up and running while I add, build, or reconfigure other machines, without the expense of all the hardware. It also brings all those machines into a single console, making it really convenient for me to manage. Along with that, I can now easily allocate more storage to the virtual machine as I have need!
Of course, this presented a problem. There is a lot of data that needed to be migrated. 1TB on the iMac, 320GB + 500GB + 160GB + 160GB on the existing media server, 160GB on the other MacBook Pro, and ~100GB on my MacBook Pro. How do you migrate all this data over? I budgeted for 1 new hard drive, but not more than one.
I purchased new server components (see below) on Boxing Day, and spent about ~$600 on what would have been an ~$830 server.
Components:
- Motherboard: Asus P5Q Pro Turbo
- CPU: Intel E7500 Boxed (mid-late 2009 versions have VT-x, a requirement for 64bit host VM’s)
- Ram: 4GB OCZ DDR2 800
- Hard-drive: WD 1.5TB
- Case: Antec P183
- PSU: Antec TruePower 650 New
- Video Card: Asus EAH4350 (Asus motherboard doesn’t have on-board video)
- NIC: Intel EXPI9301CT (Required, as ESXi 4 didn’t support the integrated Atheros-based NIC)
I began the new ESXi build by assembling the new hardware. As I wasn’t installing from USB and instead opted for the lower-hassle CD installation method, I borrowed a CD drive from my spare hardware box (I also borrowed keyboard/mouse/monitor). When I tried to boot ESXi for the first time, it informed me that the integrated NIC was not compatible. I nipped out to Canada Computers and purchased a new one, and the installation proceeded smoothly from that point on. Once installation was completed, I enabled SSH to ESXi by doing the following:
- alt-f1 (Note: As pointed out below, you will not see your typing on this screen, just trust us, it is there).
- unsupported
- root pw
- vi /etc/inetd.conf
- delete the “#” from ssh
- services.sh restart
I rebooted the machine, and went to my Windows box (no Mac ESXi/vSphere client yet) to manage the server. (If you haven’t installed the vSphere client yet, you can download it by going to your ESXi installation’s IP in your web browser) I setup a new VM for Ubuntu Linux 9.10, allocating 16GB to the VM hard drive, and configuring it with thin provisioning (dynamically expands). All my data would be on a second virtual drive. I then proceeded to install Ubuntu and install all the updates. Now it was time to add space for my data.
It was at this point that I hit a snag. VMware ESXi only supports 256GB virtual disks. I have terabytes of data, so how am I supposed to store all this information?
Enter LVM (Logical Volume Manager). LVM is a great technology that has seen a lot of use in Linux, allowing you to create volumes that span multiple hard-drives/partitions. I added 5 256GB thin provisioned disks to the Ubuntu installation, and proceeded to mount them as a single ext3 volume under /mnt/Shares. I set the volume to auto-mount on reboot in my /etc/fstab file, and proceeded to create the directories inside /mnt/Shares that would serve up my data. You can see a list of my shares below.
- Backup (To be used for Time Machine)
- Data (To be used for miscellaneous data, like eBooks)
- Movies
- Music
- Photos (Will contain an rsync-ed folder of my Adobe Lightroom photos, as well as a folder for miscellaneous images (scans, etc))
- Software (Will contain a copy of my software. I have directories created Windows and Mac, and the respective software in each)
- TV
The next interesting bit. How to actually share these shares. Obviously I don’t want to just open everything up to the world, and I also happen to have a multi-platform environment here, with Windows clients, Mac clients, and my landlord’s Linux clients. I elected to share all the shares using AFP (for my Macs), and the Movies, TV, and Software directories via SMB (for my Windows clients and the Linux clients). I created a user account called ‘main’ for my use, ‘kidd’ for my landlord, and ‘visitor’ for any guests that may be over. All three are password protected. ‘main’ has access to all shares, ‘kidd’ has access to Movies, TV, and Software, and ‘visitor’ has access to Movies and TV.
I shared the AFP shares to my Macs using a custom-compiled version of netatalk, following the great tutorial here. SMB configuration was fairly straightforward, and everything went well. I copied over as much data as possible (from the old server), before running out of disk space. At this point, I copied over data from the iMac 1TB into the now-empty old server, and popped the 1TB drive into the server. At this point I ran into the second snag. ESXi did not want to format the drive! I did some digging, and found that it was due to having been formatted as GUID before (in the Mac). I used dd to wipe out the first and last sectors of the drive, and tried to create a new partition via fdisk as per these instructions. Again, no go. After several attempts, I discovered that immediately after creating the partition table with fdisk, I had to reboot before using the vmkfstools command to create a datastore. After rebooting the server, I was able to format the drive appropriately. I logged into the Windows GUI console and deleted the new datastore. I then added the drive to the existing datastore, and fired up Ubuntu. I created several more 256GB thin-provisioned disks, and added them to the LVM.
I’ve installed ‘forked-daapd’ as a service on Ubuntu. This allows me to open Front Row and select the server as a source. I’ve also ‘tightvncserver’ set to run on startup, and have advertised the service in Avahi/Bonjour. This allows me to see the ‘Share Screen…’ button in OS X finder, and control the MediaServer via Screen Sharing, rather than a dedicated VNC client.
It’s been a great experiment, and now a great production home-media server. I’m looking forward to setting up my lab environment in other VM’s on ESXi. Feel free to leave me any questions/comments below that you might have on the process.
Related posts:
“VMware ESXi only supports 256GB virtual disks”
This is NOT true, I have plenty of 1TB and even 1.5TB disks.
You are correct Anton. However, in order to actually run that size of disk, you need to specify a larger block size for the disk format. I was referring to the default block size of 1MB, which limits the virtual disks to 256GB each.