Lessons learned for ESXi home users

With apologies for rambling, the following is a collection of “lessons
learned”, garnered over the last two years while employing ESXi 4.x in
a home network.

Online file storage

NFS is extremely slow when run in a VM (think FreeNAS in a VM). Only
thing slower is connecting USB storage to a VM. Other storage
protocols are tolerable but they are noticeably slower. Note: this is
not to say that it should never be done. Sometimes it's unavoidable.
For us home users, it’s not uncommon to have FreeNAS running in a VM.

You get what you pay for

When you build our a server, it's better to use a new system than
reuse an existing one. However, the you-get-what-you-pay-for rule
applies. Buying the low end vanilla box, and adding non-standard
drivers, is likely to end in pain and sadness.  My HP a1540n has
chugged along for 2 years without complaint.  The eMachines
EL-1352-07e died a noisy death, involving a number of lockups and
PSODs.

The idea had been that I could replace the 5+ year old machine with
one with equitable specs and smaller size.  I'd save $700 and have a
server that was small enough that it could travel to conferences.  I'm
hoping that it's just a power supply issue.

Make backups

Always (always!) make a backup before making any changes. This even
applies to simple patching and updates. It especially applies if you
experimenting with software, even more so if that software is
packaged. All it takes is one wrong dependency and some of your
installed software either disappears or ceases to function. Making a
backup is easy, though it may take a little time. A 100 GB SATA
disk-to-SATA disk backup can take about 90 minutes to create but it's
less time than having to recover or rebuild inadvertently destroyed
data.

Don’t use snapshots

Snapshots should never be used in production environments. Snapshots
can cause your VM to run slower, especially when you have multiple
large snapshots.  I'm of the belief that snapshots can remove any
speed advantage gained by using paravirtualization.

If you use snapshots and need to export a VM for any reason, there's
extra work involved in merging all snapshots back into their parents.
There are no tools, outside of the vSphere client, that handle ESXi
snapshots.  You need the flat file before you can export the VM to
some other hypervisor.  This is done by using the scary sounding
"Delete All" button in the snapshot manager.  What it actually does is
merge all snapshots back into the core disk, by merging snapshot #3
into #2, #2 into #1, and then #1 into the core.  For large VMs,
merging can require an obscene amount of storage (a couple TB of
storage can be consumed quickly).

Use scratch VMs
Always install software in a test VM before installing the software in
a production VM, especially when handling packaged software with
dependencies.  You never know what you'll break.  Example:
KnowledgeTree requires the Zend server package to provide PHP vice the
standard PHP package.  Installing anything that requires the normal
PHP package breaks KnowledgeTree.  (Note: this is also a support for
the "Make backups" recommendation.)

Know your tools
Finally, become familiar with your tools before you need them.  Think
of it as continuity planning.  It minimizes anxiety.  If you're having
to look for tools to handle a problem, after the problem has already
occurred, you'll probably use the first cheesy tool that you can find,
vice the proper tool.

Hope this helps.