Cannot install the vCenter agent service. Unknown installer error

The other I was playing around with my home lab and I was facing a weird problem when adding an ESXi4 host to vCenter. The task stopped at 19% and in the Task Status I could read “Cannot install the vCenter agent service. Unknown installer error“.

Oops, unknown installer error!? How am I going to find out what the problem is without an error code?
In the Task Details there was no useful info and no error code neither.

I searched VMware KB and found something very similar: Adding ESX host to VirtualCenter fails at 19%. I say very similar because the task also hangs at 19% but the similarity stops here as well. No evidence of a signature problem in my case!

Then I had a look at the ESXi4’s logs and the first one to look at is: /var/log/vmware/hostd.log

~ # cat /var/log/vmware/hostd.log | grep AgentMgr
[2010-04-23 23:58:50.138 3B081B90 verbose ‘AgentMgr’] Invoking command ‘/tmp/vmware-root/ha-agentmgr/upgrade’
[2010-04-23 23:56:23.193 3B081B90 warning ‘AgentMgr’] Upgrade command failed with status 201
[2010-04-23 23:56:23.193 3B081B90 warning ‘AgentMgr’] === BEGIN COMMAND OUTPUT ===
[2010-04-23 23:56:23.194 3B081B90 warning ‘AgentMgr’] [379093] 2010-04-23 23:58:50: Logging to /var/log/vmware/vpx-iupgrade.log
[2010-04-23 23:56:23.194 3B081B90 warning ‘AgentMgr’] === END COMMAND OUTPUT ===
[2010-04-23 23:57:29.783 16927B90 verbose ‘AgentMgr’] Invoking command ‘/tmp/vmware-root/ha-agentmgr/upgrade’
[2010-04-23 23:57:54.734 16927B90 warning ‘AgentMgr’] Upgrade command failed with status 201
[2010-04-23 23:57:54.735 16927B90 warning ‘AgentMgr’] === BEGIN COMMAND OUTPUT ===
[2010-04-23 23:57:54.735 16927B90 warning ‘AgentMgr’] [379578] 2010-04-23 23:57:29: Logging to /var/log/vmware/vpx-iupgrade.log
[2010-04-23 23:57:54.735 16927B90 warning ‘AgentMgr’] === END COMMAND OUTPUT ===

Much much more interesting entries here. We have confirmation of an additional log for HA management tasks such upgrades: /var/log/vmware/vpx-iupgrade.log plus we have an error code “Upgrade command failed with status 201

FYI /var/log/vmware/vpx-rupgrade.log is the log created when removing a host from vCenter.

~ # tail  /var/log/vmware/vpx-iupgrade.log
[379578] 2010-04-23 23:57:51: status = 0
[379578] 2010-04-23 23:57:51: exec /sbin/vmkramdisk vpxa.vgz Failed to mount archive: Out of memory
[379578] 2010-04-23 23:57:53: status = 12
[379578] 2010-04-23 23:57:53: exec rm -f vpxa.vgz
[379578] 2010-04-23 23:57:53: status = 0
[379578] 2010-04-23 23:57:53: exec rm -f VMware-vpx-eesx-1-linux-208111.tar.gz
[379578] 2010-04-23 23:57:53: status = 0
[379578] 2010-04-23 23:57:53: Build: 208167 Installing tarball..
[379578] 2010-04-23 23:57:53: All done! Status: 201 <- that’s our error code returned to hostd.log but unfortunately not to the Task Detail window in vCenter.

Further more, now we have the reason why it fails! Failed to mount archive: Out of memory

I’ve told you at the beginning of this post, this issue happened in my home lab. I run short on memory on my lappy thus I have assigned only 2048MB to my nested ESXi4 host which also runs two virtual machines; a Windows DC with 512MB allocated and my vCenter server with 1512MB allocated. Although 2048MB is the minimum required for an ESXi4 to boot up (see ESXi Installable and vCenter Server Setup Guide), still it’s not enough to install HA agent when you are heavily overcommitting on memory as I do. Adding a fair amount of memory (+1GB) the HA agent could finally get installed.

what if I set the ESXi4 host back to 2048MB of memory? Well it works. The HA agent starts and the host is manageable through my vCenter VM. Hurray!

At the same time I’ve added another ESXi4 host to my vCenter server VM. That host has also 2048MB allocated but had no VM running thus no memory starvation. The HA installation worked flawlessly.

Conclusion: logs, logs, logs!  Always look at the logs to troubleshoot issues you may have with your VMware environment! ESX Server 3 Log Map is a very good document describing in details all of the logs available on ESX Server 3. ESXi4 uses the same log files mapping.  N.B. As you may know log files doesn’t survive a reboot of an ESXi host because VMkernel uses a simple in-memory file system to hold the ESXi configuration files, log files, and staged patches but I’ll blog more on that next time.

Happy troubleshooting!


About PiroNet

Didier Pironet is an independent blogger and freelancer with +15 years of IT industry experience. Didier is also a former VMware inc. employee where he specialised in Datacenter and Cloud Infrastructure products as well as Infrastructure, Operations and IT Business Management products. Didier is passionate about technologies and he is found to be a creative and a visionary thinker, expressing with passion and excitement, hopefully inspiring and enrolling people to innovation and change.
This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to Cannot install the vCenter agent service. Unknown installer error

  1. Diego says:

    Excelent Post!, very helpfull.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s