Nagios vSphere monitoring via Nagios (NRPE) - Virtualization ...

Nagios vSphere monitoring via NRPE | Author: Paul Drangeid http://www.tdonline.com/training/ 

Nagios vSphere 

monitoring via NRPE 

revision 2.1 

presented by: 

Introduction: 

This document attempts to explain how to configure monitoring for your vSphere/ESX host to track 

various performance metrics (and datastore utilization) with Nagios. The concepts should be applicable 

to any Nagios implementation; however the specific instructions and screenshots included were using 

Groundwork Community Edition. These methods were specifically tested against GWCE version 6.01 on 

CentOS. The version of ESX used was vSphere 4.0, 4.1. 

©2010, TeleData Consulting, Inc. Page 1


Prerequisites: 

Working knowledge of the console and VI Client for vSphere 4.x 

Working knowledge of the console and configuration interface of your Nagios server. 

Properly installed, configured, tested and running Nagios Server. 

Ability to work with vi/nano or other Linux editor 

Goals: 

Our end-result should be monitoring of load, CPU, availability, and multiple instances of your VMFS 

datastore volumes on one or more ESX/vSphere hosts. Trending/graphing of the utilization and alerting 

based on warning and critical thresholds supplied. 

About the Author: 

Paul Drangeid is a senior systems architect and owner for TeleData Consulting, Inc. 

Began the IT career in 1994; Areas of competence have included the following focus areas: 

Virtualization (VMware ESX, Capacity Planning, SRM, XenServer, HyperV, automated deployments) 

Storage (Shared SCSI, iSCSI, SAN, DAS, NAS, replication) 

Microsoft (SQL Server, Exchange, Active Directory, Terminal Services, general infrastructure) 

Citrix (Winframe – Xenapp; Web Interface, Secure Gateways) 

Resources and Tools used: 

Nagios GroundWork OpenSource 

Community Edition 

http://sourceforge.net/projects/gwmos/files/ 

installable (BIN) and Vmware appliance versions available as 

free downloads. 

http://www.veeam.com/vmware-esxi-fastscp.html 

SCP Veeam FastSCP used to copy 

files to/from hosts 

Check_ Modified linux script Original check_vmfs shell script can be found here: 

vmfs 

http://www.yellow-bricks.com/2008/01/21/checking-thediskspace-on-your-vmfs-volumes/ 

NRPE fping 

http://www.rpmfind.net/ 

and perl-Socket6 

The latest versions of the Nagios (NRPE) daemon and 

prerequ perl-Digest-SHA1 

prerequisites can be found on rpmfind.net 

isites perl-Digest-HMAC 

perl-Crypt-DES 

Remember that vSphere is now a x64 console, so get the 

perl-Net-SNMP 

nagios-plugins 

nagios-nrpe 

x86_64 flavors of these rpms 

bundle All required scripts and RPMs 

used in this document 

http://www.tdonline.com/download/ nrpe_bundle.zip 



STEP 1) Copy the NRPE and pre-requisites to a temp 

folder on your vSphere host. 

Using Veeam FastSCP -- You will need to create a new shell (local) user on the ESX host 

if you have not already. DO this by connecting directly to the ESX host with the vSphere client, 

add the user (be sure to check the “grant shell access” checkbox) Then create the connection 

profile in Veeam, use your newly created local user account. 

in vSphere 4.0 the “Add account to the sudoers file automatically” will fail. You will need to first 

edit your sudoers file by ruuning visudo at the console. Comment out the ‘Defaults 

requiretty by prefacing it with a # 

Save changes and restart SSHD 

service sshd restart 



STEP 2) Upload files to vSphere Host: 

Create a new folder: /tmp/nrpe 

Copy the RPM files, XML file, and check commands to 

/tmp/nrpe 



STEP 3) Install NRPE agent: 

Log into the console (SSH), navigate to /tmp/nrpe and 

install the rpms: 

The order of installation is important: 

rpm -i fping-2.4-1.b2.2.el3.rf.x86_64.rpm 

rpm -i perl-Socket6-0.20-1.el3.rf.x86_64.rpm 

rpm -i perl-Digest-SHA1-2.11-1.el3.rf.x86_64.rpm 

rpm -i perl-Digest-HMAC-1.01-1.rhel2.1.dag.noarch.rpm 

rpm -i perl-Crypt-DES-2.05-3.1.el3.rf.x86_64.rpm 

rpm -i perl-Net-SNMP-5.2.0-1.1.el3.rf.noarch.rpm 

rpm -i nagios-plugins-1.4.11-1.el3.rf.x86_64.rpm 

rpm -i nagios-nrpe-2.5.2-1.el3.rf.x86_64.rpm 

############## Allow NRPE traffic through the firewall 

mv NRPE.xml /etc/vmware/firewall/ 

cd /etc/vmware/firewall 

esxcfg-firewall -e NRPE 

####### configure NRPE to auto start at reboot 

/sbin/chkconfig --level 345 nrpe on 

############## Move check commands to NRPE plugin directory 

mv check_vmfs /usr/lib64/nagios/plugins/ 

mv check_mem.pl /usr/lib64/nagios/plugins/ 

You must grant execute permissions for user nagios to the check_vmfs and 

check_mem.pl (veeam makes this easy, right click: select properties, change 

user from root to nagios, grant execute). 

If you forget this step, the NAGIOS checks will return 

an error: NRPE: Unable to read output 



STEP 4) Allow NAGIOS access to executables: 

Add commands to sudoers by executing visudo 

[root@tdesx1 nagios]# visudo 

add the following lines: 

#Allows nagios user access to nagios plugins and vdf (for vmfs checks) 

nagios ALL=NOPASSWD: /usr/lib64/nagios/plugins/ 

nagios ALL=NOPASSWD: /usr/sbin/vdf 

You would follow these same steps for any other commands that run from NRPE that require root access 

to execute We have included all check commands in the plugins subfolder, and the vdf binary (used by 

check_vmfs) 

Be aware there are security ramifications of allowing command arguments from NRPE!! Since you are 

elevating the nagios user to root, there is exposure, as someone could PIPE an extra command via the 

command line arguments from your Nagios server. You may want do some/all of the following: 

� Instead of the entire /plugins folder, only allow SUDO for specific plugins 

� Use a password for NRPE communication 

� use a nonstandard port for NRPE traffic. 

� limit allowed hosts and specify only your nagios server 

The default configuration allows you to control host access via the nrpe.cfg file (in /etc/nagios) 



STEP 5) Customize NRPE configuration: 

Edit /etc/nagios/nrpe.cfg with vi or nano 

to allow command line arguments you must change 

dont_blame_nrpe=1 

under commands add the following lines: 

command[check_swap]=sudo /usr/lib64/nagios/plugins/check_swap -w $ARG1$ -c $ARG2$ 

command[check_vmfs]=sudo /usr/lib64/nagios/plugins/check_vmfs “$ARG1$” $ARG2$ $ARG3$ 

command[check_mem]=sudo /usr/lib64/nagios/plugins/check_mem.pl -U -w $ARG1$ -c $ARG2$ 

check_vmfs uses a command that requires elevated permissions, so you will need to allow the nagios 

user to sudo when running this check command: 

Be sure to restart the nrpe client after any config changes: 

[root@tdesx1 nagios]# service nrpe restart 

1. Copy check commands to vSphere host into 

/usr/lib64/nagios/plugins/ 

You can use SCP to copy the check commands, or just use an editor and paste the command text. 



Be sure you make the command executable with chmod or edit the properties with the SCP 

GUI. 

Copy or Create the following 2 commands: 

check_vmfs 

check_mem.pl 

check_vmfs is an adaptation (modified to be parsed properly for nagios performance details to allow 

for MRTG graphs) of a script posted by Duncan Epping at Yellow Bricks: Here is the text of 

check_vmfs: 



#!/bin/bash 

STATE_OK=0 

STATE_WARNING=1 

STATE_CRITICAL=2 

STATE_UNKNOWN=3 

MYVOL=$(echo "$1" | sed "s/^$[\"']$$.*$\1\$/\2/g") 

WARNTHRESH=$2 

CRITTHRESH=$3 

VOLMATCH=-1 

myresult=sudo /usr/sbin/vdf -h -P "/vmfs/volumes/$MYVOL" | grep -E 

'^/vmfs/volumes/' | awk '{ print;}' | while read output ; do 

DISKSIZE=$(echo $output | awk '{ print $2 }' ) 

DISKUSED=$(echo $output | awk '{ print $3 }' ) 

DISKAVAILABLE=$(echo $output | awk '{ print $4 }' ) 

PERCENTINUSE=$(echo $output | awk '{ print $5 }' ) 

VOLNAME=$(echo $output | awk '{ print;}' | cut -d"%" -f2 | cut -d"/" -f4) 

CUTPERC=$(echo $PERCENTINUSE | cut -d'%' -f1 ) 

if [ "$MYVOL" = "$VOLNAME" ] ; then 

VOLMATCH=1 

if [ $CUTPERC -lt $WARNTHRESH ] ; then 

echo "OK - $PERCENTINUSE used | Volume=$1 Size=$DISKSIZE Used=$DISKUSED 

Available=$DISKAVAILABLE PercentUsed=$PERCENTINUSE" 

exit 7 

fi 

if [ $CUTPERC -ge $CRITTHRESH ] ; then 

echo "CRITICAL - *$PERCENTINUSE used* | Volume=$1 Size=$DISKSIZE 

Used=$DISKUSED Available=$DISKAVAILABLE PercentUsed=$PERCENTINUSE" 

exit 2 

fi 

if [ $CUTPERC -ge $WARNTHRESH ] ; then 

echo "WARNING - *$PERCENTINUSE used* | Volume=$1 Size=$DISKSIZE 

Used=$DISKUSED Available=$DISKAVAILABLE PercentUsed=$PERCENTINUSE" 

exit 1 

fi 

fi 

if [ $VOLMATCH -ge 1 ] ; then 

break 

fi 

done 

RET=$? 

if [ $RET -le 0 ] ; then 

echo "WARNING - $MYVOL not found* -- Check spelling (case matters) and that 

the datastore is mounted." 

exit 3 

fi 

if [ $RET -ge 7 ] ; then 

exit 0 

fi 



6)Configure Nagios Commands and Services 

Login to your Nagios Server. 

Navigate to Configuration: 

Click Commands. 

Either Copy an existing command or Create a new one. Call it check_nrpe_vmfs: 

The 3 arguments will represent the following: 

ARG1 – the name of your VMFS volume (this is case sensitive!) 

ARG2– the warning threshold (percent) used space at which an alert is raised 

ARG3 – the critical threshold (percent) used space at which an alert is raised 

To test the command you must get the exact name of your VMFS volume. To check this open your 

vSphere Client and select your host. Click the Configuration Tab, the click Storage: 



Go into the command configuration and test the check command as follows: 

Note in the above an error: Received 0 bytes. Go double check the steps above: 

Does the check_vmfs have proper execute permissions? 

Did you setup the sudo user properly? 

Did you restart the xinetd, inetd, or nrpe client after making configuration changes? 

Do you need to supply an alternate port, or NRPE password on the command line? 

Once it is working you should see a result like follows: 



“Save” the Command. 

The “OK – 25% used” indicates that the used space is below our warning threshold (45%). The info after 

the pipe “|” is the performance data that we can use for graphing (we’ll get to that later). 

Now that the command works, let’s setup the service: 

Either “create new”, or clone a service. Name the service vSphere_vmfs. Click on the “Service Check” 

tab to specify the command line specifics: 

Be sure the check command is check_nrpe_vmfs 

Validate the command line prompts for 3 arguments. 



Save your service. Edit your Host, select your ESX Server, and add the new service to the host: 

After you click Add you should see the service listed: 

Now let’s add your various VMFS instances you would like to track. Go to Hosts - -> 

vSphere_vmfs 



On the right hand screen click Service Check to alter the instance specifics: 



Add your VMFS volumes (precede the Instance Name with an underscore “_”. The Arguments need to 

be !VMFS-VOLUME-NAME!WarningThresh!CriticalThresh 

When you are finished click save. 



You should now test your arguments at the top of this screen. Paste the arguments into the Command 

Line box, and click Test. Verify that you properly get results for your check: 

Go back to the Service Details Screen, and verify that The Extended info template is set to 

“number_graph” – this will create a link in order to display a graph of historical data for this service 

(we’ll setup performance graphing next): 



Now you should perform your Preflight Test and Commit: 

Now verify that you are getting results from the monitoring (go to your status screen): 

In order to receive graphing data we must configure performance collection: 



Click Configure and Create a new Entry (or better yet Copy and existing one and change the data: 

Note the trailing underscore in the service name. It is crucial that you match this name exactly to the 

service created above. This will allow you to graph all instances of this service automatically, as long as 

the instance is _instancename 

Graph Label: VMFS Usage (pct) 

Service: vSphere_vmfs_ 

Use Service as a 

Regular 

Expression 

Host: * 

Status Text 

Parsing Regular 

Expression: 

Use Status Text 

Parsing instead 

of Performance 

Data 

ON 

OFF 

RRD Name /usr/local/groundwork/rrd/$HOST$_$SERVICE$.rrd 

RRD Create 

Command 

RRD Update 

Command 

Custom RRDtool 

Graph Command 

Enable ON 

$RRDTOOL$ create $RRDNAME$ --step 300 --start n-1yr 

$LISTSTART$DS:$LABEL#$:GAUGE:1800:U:U$LISTEND$ RRA:AVERAGE:0.5:1:8640 

RRA:AVERAGE:0.5:12:9480 

$RRDTOOL$ update $RRDNAME$ -t $LABELLIST$ $LASTCHECK$:$VALUELIST$ 2>&1 

'rrdtool graph - DEF:a="rrd_source":ds_source_4:AVERAGE AREA:a#33FFFF 

AREA:a#3399FF:"Datastore Utilized\:" GPRINT:a:LAST:"%.2lf Percent" 

GPRINT:a:MIN:min=%.2lf GPRINT:a:AVERAGE:avg=%.2lf GPRINT:a:MAX:max="%.2lf\l" - 

c BACK#FFFFFF -c CANVAS#FFFFFF -c GRID#C0C0C0 -c MGRID#404040 -c 

ARROW#FFFFFF -Y -u 100 -l 0 --rigid' 

Once you are getting data and have performance collection properly configured it may (and will likely) 

take 20-30 minutes before graph data begins to appear in the Status (or Nagios details) views. Once you 

see the graph with a data line you can uncork the Chablis! You are now graphing your ESX VMFS 

utilization trends! 



Assuming you have notifications and email properly configured on your Nagios you should be receiving 

notification if you exceed your specified utilization thresholds. 



Here’s the Config for the Memory Graph: 

Graph Label: vSphere Memory Utilization 

Service: vSphere_memory 

Use Service as a 

Regular 

Expression 

Host: * 

Status Text 

Parsing Regular 

Expression: 

Use Status Text 

Parsing instead of 

Performance Data 

ON 

OFF 

RRD Name /usr/local/groundwork/rrd/$HOST$_$SERVICE$.rrd 

RRD Create 

Command 

RRD Update 

Command 

Custom RRDtool 

Graph Command 

Enable ON 

$RRDTOOL$ create $RRDNAME$ --step 300 --start n-1yr $LISTSTART$ 

DS:$LABEL#$:GAUGE:1800:U:U DS:$LABEL#$_wn:GAUGE:1800:U:U 

DS:$LABEL#$_cr:GAUGE:1800:U:U $LISTEND$ RRA:AVERAGE:0.5:1:8640 

RRA:AVERAGE:0.5:12:9480 

$RRDTOOL$ update $RRDNAME$ $LASTCHECK$:$VALUE1$:$WARN1$:$CRIT1$ 2>&1 

'rrdtool graph - DEF:a="rrd_source":ds_source_0:AVERAGE 

DEF:w="rrd_source":ds_source_1:AVERAGE 

DEF:c="rrd_source":ds_source_2:AVERAGE CDEF:cdefa=a CDEF:cdefb=a,0.99,* 

CDEF:cdefw=w CDEF:cdefc=c CDEF:cdefm=c,1.05,* AREA:a#33FFFF 

AREA:cdefb#3399FF:"Memory Utilized\:" GPRINT:a:LAST:"%.2lf Percent" 

GPRINT:cdefa:MIN:min=%.2lf GPRINT:cdefa:AVERAGE:avg=%.2lf 

GPRINT:cdefa:MAX:max="%.2lf\l" LINE2:cdefw#FFFF00:"Warning Threshold\:" 

GPRINT:cdefw:LAST:"%.2lf" LINE2:cdefc#FF0033:"Critical Threshold\:" 

GPRINT:cdefc:LAST:"%.2lf\l" COMMENT:"Service\: SERVICE" 

CDEF:cdefws=a,cdefw,GT,a,0,IF AREA:cdefws#FFFF00 CDEF:cdefcs=a,cdefc,GT,a,0,IF 

AREA:cdefcs#FF0033 CDEF:cdefwt=a,cdefw,GT,cdefw,0,IF LINE:cdefwt#000000 

CDEF:cdefct=a,cdefc,GT,cdefc,0,IF LINE:cdefct#000000 -c BACK#FFFFFF -c 

CANVAS#FFFFFF -c GRID#C0C0C0 -c MGRID#404040 --vertical-label="Used (pct)" -title="vSphere 

Memory Usage" -c ARROW#FFFFFF -Y -u 100 -l 0 --rigid'

Nagios vSphere monitoring via Nagios (NRPE) - Virtualization ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?