ESXi 4.1 – Major Security Issue – The Sequel And The Workaround


Twitter is good! Twitter is great! Today, late in the evening my time, I started to read tweets about a major security issue within ESX4.1.0. It all started with a post from nirvy in a VMware forum followed by an article and a tweet from lamw that I encourage you to read at virtuallyGhetto.com before you keep on reading my post.
I have tested the issue in my home lab and I can confirm the problem with ESXi4.1.0 fresh install or upgrade! I could  not resist to try to find out the root cause of the problem or at least a fix/workaround.  After a couple of hours here are what I have found:

  • ESX4.1 doesn’t use MD5 but DES – cat /etc/shadow -> hash for root’s password doesn’t start with ‘$1$‘ and hash is very short.
  • /etc/pam.d/common-password doesn’t exist but VMware KB1012033 tells you to edit that file if you want to change password restriction settings.
  • sshd uses /etc/pam.d/system-auth file but that file has changed a lot since ESX4.0.

I leave to VMware to create a fix to address the issue. In the mean time here is a workaround to force MD5 encryption of root’s password but first the reminder!
This is an UNSUPPORTED hack that can lead to UNSTABLE system! Don’t try this on a production environment (don’t even think about it) and read the disclaimer below! I shall not be liable for any damages arising out! Now that I have scared you let’s move on with the fix.

  1. vi /etc/pam.d/system-auth and change this line accordingly: password sufficient /lib/security/$ISA/pam_unix.so use_authtok nullok md5 shadow
  2. save and exit the file with :wq!
  3. change your password in DCUI,
  4. verify that it has encrypted your root’s password using MD5 algorithm: cat /etc/shadow <- If root’s password hash starts with a ‘$1$‘ then MD5 algorithm was used.

[UPDATE1] Instead of using CHMOD, it is much easier and safer to  issue a :wq! to save and exit the file. Mike Horwath notified me my error. I spent too much years on Windows doh! Thx Mike!
Unfortunately the change doesn’t survive a reboot of the host and /etc/pam.d/system-auth file gets reverted back to default file. That means if you change again the root password from DCUI for instance, it will be again encrypted with a DES algorithm.
My feeling this issue is related somehow to Likewise Identity Service licensed by VMware in ESX(i)4.1.0 but that’s just a thought!
Now do we really have a major security problem here, something like a Microsoft zero-day virus or attack? It is likely that you’re not at risk but anyway VMware should come back with a fix to force MD5 by default and let me tell you why. At the moment the encryption uses Data Encryption Standard (DES) which was cracked in the late 1990’s. The problem with DES is that it is a 56bit algorithm, and thus can produce a maximum of 2^56 unique hashes. It is indeed a weak algorithm in terms of strength. The mitigating factor is that you need to have access to /etc/shadow to be able to crack the hashes.

[UPDATE2] I’ve mixed up two different issues! DES and LM HASH.  DES definitely truncates after 8 characters, there is no ‘second hash’. This is by design. On the other hand LM HASH does use a second hash for anything beyond 7 characters. I should have double checked instead relying solely on my memory… Thx Bert, I owe you one🙂

[UPDATE3] VMware just published KB1024500 describing the behavior but so far no patch foreseen to encrypt passwords using MD5 instead of DES.

[UPDATE4] VMware just updated its KB1024500 with a solution to the issue. The steps are identical to those I detailed above. VMware goes further with an additional step for ESXi only to allow the change to persist through reboots. Now we are waiting for the patch… Great work VMware!

[UPDATE5] VMware updated again its KB! Now VMware is expecting to release a patch to this issue. Stay tuned!

[UPDATE6] November 15th 2010, VMware just released a patch for bug fixes and especially for the password truncated at eight characters, read more at VMware ESXi 4.1, Patch Release ESXi410-201010001

[UPDATE7] VMware just published a KB titled Changing the default password hashing algorithm from md5 to sha512 in ESX 4.x. The KB basically describes the same process, that is editing /etc/pam.d/system-auth to append a SHA512 instead of MD5. That’s an overall improvement of the security. Thx VMware.


DISCLAIMER. THIS INFORMATION IS PROVIDED TO YOU “AS IS” WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, WHETHER ORAL OR WRITTEN, EXPRESS OR IMPLIED. THE AUTHOR SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OR CONDITIONS OF MERCHANTABILITY, SATISFACTORY QUALITY, NON-INFRINGEMENT AND FITNESS FOR A PARTICULAR PURPOSE AND SHALL NOT BE LIABLE FOR ANY DAMAGES ARISING OUT OF OR IN CONNECTION WITH THE USE OF THIS CONTENT, INCLUDING DIRECT, INDIRECT, CONSEQUENTIAL DAMAGES, LOSS OF BUSINESS PROFITS OR SPECIAL DAMAGES, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

About PiroNet

Didier Pironet is an independent blogger and freelancer with +15 years of IT industry experience. Didier is also a former VMware inc. employee where he specialised in Datacenter and Cloud Infrastructure products as well as Infrastructure, Operations and IT Business Management products. Didier is passionate about technologies and he is found to be a creative and a visionary thinker, expressing with passion and excitement, hopefully inspiring and enrolling people to innovation and change.
This entry was posted in Uncategorized. Bookmark the permalink.

16 Responses to ESXi 4.1 – Major Security Issue – The Sequel And The Workaround

  1. Pingback: Tweets that mention ESXi 4.1 – Major Security Issue – The Sequel And The Workaround « DeinosCloud -- Topsy.com

  2. Mike Horwath says:

    You linked http://www.virtuallyghetto.com/2010/07/esxi-41-major-security-issue.html

    I had posted there that it was a DES issue.

    Also, you don’t need full read/write/execute permissions to edit the file, in fact, as root, you can easily save it via:

    :wq!

    which is colon, double-u, queue, exclamation point

    means: write, quit, and do it cause I said so.

    Someone forgets to fix the permissions of the file could cause some security issues further down the line, no disclaimers will help them😦

    Can’t wait for VMware to fix this – not like DES was old and cold in the 1990s…

  3. NiTRo says:

    Or you can use AD integration as workaround🙂

  4. Pingback: VMware vSphere - виртуализация ЦОД » Баг с Security в VMware ESXi 4.1.

  5. Andrew Whalley says:

    Great work. Though please note that MD5 is also “considered cryptographically broken and unsuitable for further use.” [1]. So hopefully a fix will use SHA-1 [2].

    [1] http://www.kb.cert.org/vuls/id/836068
    [2] Yep, I know that has weaknesses too.

    • DK says:

      SHA-1 is very vulnerable to collisions. Computational complexity of cracking SHA-1 hashes is very low (2^59). MD5 higher then 2^100. Rainbow tables are a big aid in MD5 cracking, but still I would rather trust MD5 more then SHA-1.

  6. MD5 Hash says:

    Thanks. Great find. DES must go.

  7. triple-DES as used in crypt() on unix/linux always truncates to 8 chars. There’s no “second hash” being generated at all. The second hash thing is something Windows did with the LMHASH method I believe ?

    • deinoscloud says:

      You damn right, I’ve been mixing up with LM HASH mechanism… Post updated!

      BTW VMware just published a KB regarding this particular issue, but it is unclear of the why the encryption has been changed to DES and when VMware will have a patch to fix it…

      Cheers,

  8. Pingback: Welcome to vSphere-land! » vSphere 4.1 - The Links

  9. The patch for this password truncate issue was just released : http://kb.vmware.com/kb/1027021

  10. Pingback: A Year Blogging In Summary And Season’s Greetings « DeinosCloud

  11. deinoscloud says:

    VMware just published a KB titled Changing the default password hashing algorithm from md5 to sha512 in ESX 4.x. (http://kb.vmware.com/kb/1032666)

    The KB basically describes the same process, that is editing /etc/pam.d/system-auth to append a SHA512 instead of MD5.

    That’s an overall improvement of the security. Thx VMware.

  12. Carlo says:

    Thanks for the update on SHA512! I’ve been waiting for so long for VmWare to adopt a more secure hash function! Can’t wait now to apply KB1032666.
    Thanks for sharing
    Carlo

  13. Pingback: ESXi 4.1 - Major Security Issue

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s