ESXi – Automatic Backups

Did you know that ESXi does an hourly backup of its configuration files? Have a look at the crontabs for root account: cat /var/spool/cron/crontabs/root

#syntax : minute hour day month dayofweek command
01 01 * * * /sbin/tmpwatch.sh
01 * * * * /sbin/auto-backup.sh #first minute of every hour (run every hour)

So auto-backup.sh is the script that is triggered hourly to backup ESXi. Let’s see what’s in there: cat /sbin/auto-backup.sh

#!/bin/ash export PATH=/sbin:/bin #PXE boot? if [ ! -d /bootbank ]; then exit 1 fi first=0 if esxcfg-info -e; then if [ ! -f /bootbank/local.tgz ]; then first=1 else mkdir -p /tmp/auto-backup.$$.dir cp /bootbank/local.tgz /tmp/auto-backup.$$.dir fi else if [ ! -f /bootbank/state.tgz ]; then first=1 else mkdir -p /tmp/auto-backup.$$.dir tar zxvf /bootbank/state.tgz -C /tmp/auto-backup.$$.dir fi fi if [ “$first” = “1” ]; then #always do the first one backup.sh 0 if [ $? -ne 0 ]; then logger “Auto-Backup failed” fi exit 0 fi tar zxvf /tmp/auto-backup.$$.dir/local.tgz -C /tmp/auto-backup.$$.dir filestosave=`find /etc -follow -type f -name “.#*” | sed ‘s/\.#\(.*\)/\1/g’` for filename in $filestosave do diff -N $filename /tmp/auto-backup.$$.dir$filename if [ $? -ne 0 ]; then backup.sh 0 if [ $? -ne 0 ]; then logger “Auto-Backup failed” fi break fi done #clean up rm -rf /tmp/auto-backup.$$.dir

What is the /bootbank mentioned in the script? The /bootbank is a 48MB partition which contains the 32MB core hypervisor code. It contains the following files:

~ # ls -1 /bootbank/

aam.vgz

boot.cfg

cim.vgz

cimstg.tgz

license.tgz

mod.tgz

oem.tgz

pkgdb.tgz

state.tgz <- that’s our backup, less than 10KB in standalone or less than 100KB when part of a HA cluster (aam)

sys.vgz

vmk.gz

vmkboot.gz

vpxa.vgz

Charu Chaubal, Technical Marketing Manager at VMware has written an excellent document about The Architecture of VMware ESXi. It’s a 10 pages white paper that worth a read!

Now for the fun let’s run the script: /sbin/auto-backup.sh

~ # /sbin/auto-backup.sh config implicitly loaded local.tgz etc/vmware/vmkiscsid/vmkiscsid.db etc/dropbear/dropbear_dss_host_key etc/dropbear/dropbear_rsa_host_key etc/opt/vmware/aam/configBackup.tar.gz etc/opt/vmware/vpxa/vpxa.cfg etc/opt/vmware/vpxa/dasConfig.xml etc/opt/init.d/vmware-aam etc/sysconfig/network etc/vmware/hostd/authorization.xml etc/vmware/hostd/hostsvc.xml etc/vmware/hostd/pools.xml etc/vmware/hostd/vmAutoStart.xml etc/vmware/hostd/vmInventory.xml etc/vmware/hostd/proxy.xml etc/vmware/ssl/rui.crt etc/vmware/ssl/rui.key etc/vmware/vmkiscsid/initiatorname.iscsi etc/vmware/vmkiscsid/iscsid.conf etc/vmware/vmware.lic etc/vmware/config etc/vmware/dvsdata.db etc/vmware/esx.conf etc/vmware/license.cfg etc/vmware/locker.conf etc/vmware/snmp.xml etc/group etc/host.conf etc/hosts etc/inetd.conf etc/chkconfig.db etc/nsswitch.conf etc/passwd etc/random-seed etc/resolv.conf etc/shadow etc/sfcb/repository/root/interop/cim_indicationfilter.idx etc/sfcb/repository/root/interop/cim_indicationhandlercimxml.idx etc/sfcb/repository/root/interop/cim_listenerdestinationcimxml.idx etc/sfcb/repository/root/interop/cim_indicationsubscription.idx — /etc/vmware/license.cfg Fri Feb 12 17:07:04 2010 +++ /tmp/auto-backup.220438.dir/etc/vmware/license.cfg Fri Fe b 12 17:06:04 2010 @@ -1,5 +1,5 @@ – AQD+yggAAABQn0ksiuhfjI8tx1W6jQAAADRvNHwiujst6g6cz1zqj PU0X5EWq2+BI2Eos/M1L97wCcL0idjmsnu2UEsC+HdKOhL99 + AQD+ygksj83gAAAC7A3ExRK/5GDQAAAA/sfbAUxyd,mfvimjsbd V1dUYVMJHzJXwHQSGvy9wQ1GsdfsKIZLyf6diroKZk8EVlDzSeval \ No newline at end of file config implicitly loaded Saving current state in /bootbank Clock updated. Time: 17:07:17 Date: 02/12/2010 UTC

You could also invoke the backup manually with the vSphere PowerCLI: Get-VMHost MyESXiHost | Set-VMHostFirmware -BackupConfiguration -DestinationPath C:\Temp
Knowing that we have an hourly backup of of ESXi host, you could also do a daily backup to a backup storage and do it for all your ESXi hosts at once. NiTRo from Hypervisor.fr came up with the following nice script to achieve that:

$backpath = “x:\<your backup storage path>\” $eVMHs = Get-View -ViewType HostSystem |?{$_.config.product.ProductLineId -eq “embeddedEsx”} |?{$_.Runtime.ConnectionState -eq “connected”} Foreach ($eVMH in $eVMHs) {Set-VMHostFirmware -VMHost $eVMH.name -BackupConfiguration -DestinationPath $backpath}

Now that we saw how to backup, let’s try a restore from vSphere PowerCLI.
Step#1, put the ESXi host is maintenance mode, this is a requirement:
set-VMHost -vmhost MyESXiHost -state Maintenance
Step#2, restore the config bundle:
Get-VMHost MyESXiHost | Set-VMHostFirmware -Restore -SourcePath C:\Temp\configBundle-MyESXiHost.tgz
You can eventually add the -Force if the bundle is mismatched, that is when you restore to a different ESXi build.

Sources: VMware.com, Virtu-al.net