Home > ESXi, vSphere > ESXi>Graylog2 – Quickstart

ESXi>Graylog2 – Quickstart


This is the English version of a blog post from Raphael Schitz at hypervisor.fr.

[UPDATE] For those who want to quickly setup alarms, you need to modify the following file: /usr/share/graylog2-web/config/email.yml and add these two lines in your crontab:

su - -c 'cd /usr/share/graylog2-web;rake RAILS_ENV=production streamalarms:send>>/var/log/graylog.log'
su - -c 'cd /usr/share/graylog2-web;rake RAILS_ENV=production subscriptions:send>>/var/log/graylog.log'

Those who are using Graylog2 know how a powerful syslog server it is. And you do know as well how painful it is to install and configure. Furthermore, those who have been using it to collect ESXi logs have noticed that Graylog2 doesn’t support ESXi 5.x log format. ESXi 4.x log format are perfectly handled though. Let’s kill two birds with one stone :)

Simplicity wise Mick Pollard posted this summer an How-To guide on how to install and configure Graylog2 packages on Ubuntu 12.04. We will be adding some pieces from another How-To guide to make the Graylog2 web interface run under apache.

For compatibility sake, we will configure Graylog2′s listener on an alternate port, 1054 in this case. Indeed port 514 will be used for rsyslog which will ingest ESXi 5.x logs and forward them in the correct format to the Graylog2 server. Attention this is going to be fast:

echo 'deb http://ppa.lunix.com.au/ubuntu/ precise main' | sudo tee /etc/apt/sources.list.d/aussielunix.list
apt-key adv --keyserver keyserver.ubuntu.com --recv D77A4DCC
apt-get update
apt-get install mongodb elasticsearch graylog2-server graylog2-web apache2 libapache2-mod-passenger

Have a snack cause there is about 500MB to download and install…

Next we will have to configure some stuff:
/etc/graylog2.conf
syslog_listen_port = 10514

/etc/rsyslog.conf
$ModLoad immark
$ModLoad imudp
$UDPServerRun 514
$ModLoad imtcp
$InputTCPServerRun 514

/etc/rsyslog.d/50-default.conf
#*.*;auth,authpriv.none -/var/log/syslog

/etc/apache2/sites-available/default

<VirtualHost *:80>
DocumentRoot /usr/share/graylog2-web/public/
RailsEnv 'production'
usr/share/graylog2-web/public/>
Allow from all
Options -MultiViews
</Directory>

ErrorLog /var/log/apache2/error.log
LogLevel warn
CustomLog /var/log/apache2/access.log combined
</VirtualHost>

/etc/rsyslog.d/32-graylog2.conf

$template GRAYLOG2,"<%PRI%>1 %timegenerated:::date-rfc3339% %HOSTNAME% %syslogtag% - %APP-NAME%: %msg:::drop-last-lf%\n"
$ActionForwardDefaultTemplate GRAYLOG2
$PreserveFQDN on
*.*     @localhost:10514

/etc/security/limits.conf
root – nofile 64000
root – memlock unlimited

/etc/pam.d/su
session required pam_limits.so

Then shake it baby :)

service elasticsearch start
service mongodb restart
service graylog2-server start
service rsyslog restart
service apache2 restart

Following the format of  the messages, you may require a reverse DNS to do hostname to IP lookups. On the screenshot below you will notice logs from ESXi 5, pfSense and Astaro/UTM. We have also validated this configuration for ESXi 4FreeNAS,NTsyslogSnare/Epilog and nxlog.

Enjoy Graylog2 great features such Streams and Analytics.

About these ads
Categories: ESXi, vSphere Tags: , , , , , ,
  1. November 1, 2012 at 15:58 | #1

    Thanks a lot for this nice work Didier!

  2. November 2, 2012 at 07:33 | #2

    In order for rsyslogd to listen to TCP 514, you have to use “$PrivDropToUser adm” in /etc/rsyslog.conf

  3. December 2, 2012 at 05:21 | #3

    Thanks for the article! It helped with configuring Apache and rsyslog on my system. It now logs for a web server, a Jabber server, and an ESXi box.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s