February 10, 2012

NFS Statistics Using Linux SNMP

Using SNMP, I have been creating NFS graphs like below for all of our NetApp Filers using RRDTool

Ever since I started running NFS on Linux servers in addition to the Netapps, I wanted to graph the Linux nfs traffic. The RedHat distro comes with net-snmp package. While the package does not have any specific MIB support for NFS, it is infinitely extensible using custom external programs. Practically anything can be pushed/pulled via SNMP. So I decided to write a perl script (well, it started out as a shell script) to convert nfsstats into SNMP values.

The result is proc2snmp.pl. In addition to reporting the nfsd statistics reported by the system, the script also reports another statistical value “v3total”, which is the total number of all NFS calls (since last time the nfsd process has started). I am using this value to calculate percentages of each type of call for graphing purposes.

proc2snmp.pl source

#!/usr/bin/perl
 
%nfsd = ( "progname" => "/usr/local/custom/proc2snmp.pl" );
$place = ".1.3.6.1.4.1.2021.255";
#
# $place.1 - rc
# $place.2 - fh
# $place.3 - io
# $place.4 - th
# $place.5 - ra
# $place.6 - net
# $place.7 - rpc
# $place.8 - proc3
# $place.9 - proc4
#
 
$req = $ARGV[1];
 
#
# -s will be passed for SNMPSET command
#
if ( $ARGV[0] eq "-s") {
	# we do not do any set ops
	print "not-writable";
	exit;
}
 
#define the nfsd array
open (OUTPUT, "/bin/cat /proc/net/rpc/nfsd |");
while (<OUTPUT>) {
	@vals = split(/ /);
	if ( $vals[0] eq "proc3" ) {
		$numvals = @vals;
		$nfsd{'v3null'} = $vals[2];
		$nfsd{"v3getattr"}  = $vals[3];
		$nfsd{"v3setattr"} = $vals[4];
		$nfsd{"v3lookup"} = $vals[5];
		$nfsd{"v3access"} = $vals[6];
		$nfsd{"v3readlink"} = $vals[7];
		$nfsd{"v3read"} = $vals[8];
		$nfsd{"v3write"} = $vals[9];
		$nfsd{"v3create"}  = $vals[10];
		$nfsd{"v3mkdir"}  = $vals[11];
		$nfsd{"v3symlink"}  = $vals[12];
		$nfsd{"v3mknod"}  = $vals[13];
		$nfsd{"v3remove"}  = $vals[14];
		$nfsd{"v3rmdir"}  = $vals[15];
		$nfsd{"v3rename"}  = $vals[16];
		$nfsd{"v3link"}  = $vals[17];
		$nfsd{"v3readdir"}  = $vals[18];
		$nfsd{"v3readdirplus"}  = $vals[19];
		$nfsd{"v3fsstat"}  = $vals[20];
		$nfsd{"v3fsinfo"}  = $vals[21];
		$nfsd{"v3pathconf"}  = $vals[22];
		$nfsd{'v3commit'} = $vals[23];
	}
}
close(OUTPUT);
 
# calculate total
$v3total=0;
foreach $val (values %nfsd) {
	$v3total += $val;
}
#shove it in the array
$nfsd{'v3total'} = $v3total;
 
#
# -n will be passed for GETNEXT request
#
if ( $ARGV[0] eq "-n" ) {
	if ( $req eq "$place") { $ret = "$place.8"; }
	elsif ( $req eq "$place.8" ) { $ret = "$place.8.1"; }
	elsif ( $req eq "$place.8.1" ) { $ret = "$place.8.2.1"; }
	elsif ( $req eq "$place.8.2.1" ) { $ret = "$place.8.2.2"; }
	elsif ( $req eq "$place.8.2.2" ) { $ret = "$place.8.3.1"; }
	elsif ( $req eq "$place.8.3.1" ) { $ret = "$place.8.3.2"; }
	elsif ( $req eq "$place.8.3.2" ) { $ret = "$place.8.4.1"; }
	elsif ( $req eq "$place.8.4.1" ) { $ret = "$place.8.4.2"; }
	elsif ( $req eq "$place.8.4.2" ) { $ret = "$place.8.5.1"; }
	elsif ( $req eq "$place.8.5.1" ) { $ret = "$place.8.5.2"; }
	elsif ( $req eq "$place.8.5.2" ) { $ret = "$place.8.6.1"; }
	elsif ( $req eq "$place.8.6.1" ) { $ret = "$place.8.6.2"; }
	elsif ( $req eq "$place.8.6.2" ) { $ret = "$place.8.7.1"; }
	elsif ( $req eq "$place.8.7.1" ) { $ret = "$place.8.7.2"; }
	elsif ( $req eq "$place.8.7.2" ) { $ret = "$place.8.8.1"; }
	elsif ( $req eq "$place.8.8.1" ) { $ret = "$place.8.8.2"; }
	elsif ( $req eq "$place.8.8.2" ) { $ret = "$place.8.9.1"; }
	elsif ( $req eq "$place.8.9.1" ) { $ret = "$place.8.9.2"; }
	elsif ( $req eq "$place.8.9.2" ) { $ret = "$place.8.10.1"; }
	elsif ( $req eq "$place.8.10.1" ) { $ret = "$place.8.10.2"; }
	elsif ( $req eq "$place.8.10.2" ) { $ret = "$place.8.11.1"; }
	elsif ( $req eq "$place.8.11.1" ) { $ret = "$place.8.11.2"; }
	elsif ( $req eq "$place.8.11.2" ) { $ret = "$place.8.12.1"; }
	elsif ( $req eq "$place.8.12.1" ) { $ret = "$place.8.12.2"; }
	elsif ( $req eq "$place.8.12.2" ) { $ret = "$place.8.13.1"; }
	elsif ( $req eq "$place.8.13.1" ) { $ret = "$place.8.13.2"; }
	elsif ( $req eq "$place.8.13.2" ) { $ret = "$place.8.14.1"; }
	elsif ( $req eq "$place.8.14.1" ) { $ret = "$place.8.14.2"; }
	elsif ( $req eq "$place.8.14.2" ) { $ret = "$place.8.15.1"; }
	elsif ( $req eq "$place.8.15.1" ) { $ret = "$place.8.15.2"; }
	elsif ( $req eq "$place.8.15.2" ) { $ret = "$place.8.16.1"; }
	elsif ( $req eq "$place.8.16.1" ) { $ret = "$place.8.16.2"; }
	elsif ( $req eq "$place.8.16.2" ) { $ret = "$place.8.17.1"; }
	elsif ( $req eq "$place.8.17.1" ) { $ret = "$place.8.17.2"; }
	elsif ( $req eq "$place.8.17.2" ) { $ret = "$place.8.18.1"; }
	elsif ( $req eq "$place.8.18.1" ) { $ret = "$place.8.18.2"; }
	elsif ( $req eq "$place.8.18.2" ) { $ret = "$place.8.19.1"; }
	elsif ( $req eq "$place.8.19.1" ) { $ret = "$place.8.19.2"; }
	elsif ( $req eq "$place.8.19.2" ) { $ret = "$place.8.20.1"; }
	elsif ( $req eq "$place.8.20.1" ) { $ret = "$place.8.20.2"; }
	elsif ( $req eq "$place.8.20.2" ) { $ret = "$place.8.21.1"; }
	elsif ( $req eq "$place.8.21.1" ) { $ret = "$place.8.21.2"; }
	elsif ( $req eq "$place.8.21.2" ) { $ret = "$place.8.22.1"; }
	elsif ( $req eq "$place.8.22.1" ) { $ret = "$place.8.22.2"; }
	elsif ( $req eq "$place.8.22.2" ) { $ret = "$place.8.23.1"; }
	elsif ( $req eq "$place.8.23.1" ) { $ret = "$place.8.23.2"; }
	elsif ( $req eq "$place.8.23.2" ) { $ret = "$place.8.24.1"; }
	elsif ( $req eq "$place.8.24.1" ) { $ret = "$place.8.24.2"; }
	else { exit 0; }
}
 
#
# Rest of the code is for SNMPGET (-g)
#
else {
	if ( $req eq "$place" ) { exit 0; }
	else { $ret = $req; }
}
 
print "$ret\n";
if ( $ret eq "$place.8.1" ) { print "string\n/usr/local/custom/proc2snmp.pl\n"; exit 0;}
if ( $ret eq "$place.8.2.1" ) { print "string\nv3null\n"; exit 0;}
if ( $ret eq "$place.8.2.2" ) { print "gauge\n$nfsd{'v3null'}\n"; exit 0;}
if ( $ret eq "$place.8.3.1" ) { print "string\nv3getattr\n"; exit 0;}
if ( $ret eq "$place.8.3.2" ) { print "gauge\n$nfsd{'v3getattr'}\n"; exit 0;}
if ( $ret eq "$place.8.4.1" ) { print "string\nv3setattr\n"; exit 0;}
if ( $ret eq "$place.8.4.2" ) { print "gauge\n$nfsd{'v3setattr'}\n"; exit 0;}
if ( $ret eq "$place.8.5.1" ) { print "string\nv3lookup\n"; exit 0;}
if ( $ret eq "$place.8.5.2" ) { print "gauge\n$nfsd{'v3lookup'}\n"; exit 0;}
if ( $ret eq "$place.8.6.1" ) { print "string\nv3access\n"; exit 0;}
if ( $ret eq "$place.8.6.2" ) { print "gauge\n$nfsd{'v3access'}\n"; exit 0;}
if ( $ret eq "$place.8.7.1" ) { print "string\nv3readlink\n"; exit 0;}
if ( $ret eq "$place.8.7.2" ) { print "gauge\n$nfsd{'v3readlink'}\n"; exit 0;}
if ( $ret eq "$place.8.8.1" ) { print "string\nv3read\n"; exit 0;}
if ( $ret eq "$place.8.8.2" ) { print "gauge\n$nfsd{'v3read'}\n"; exit 0;}
if ( $ret eq "$place.8.9.1" ) { print "string\nv3create\n"; exit 0;}
if ( $ret eq "$place.8.9.2" ) { print "gauge\n$nfsd{'v3create'}\n"; exit 0;}
if ( $ret eq "$place.8.10.1" ) { print "string\nv3mkdir\n"; exit 0;}
if ( $ret eq "$place.8.10.2" ) { print "gauge\n$nfsd{'v3mkdir'}\n"; exit 0;}
if ( $ret eq "$place.8.11.1" ) { print "string\nv3write\n"; exit 0;}
if ( $ret eq "$place.8.11.2" ) { print "gauge\n$nfsd{'v3write'}\n"; exit 0;}
if ( $ret eq "$place.8.12.1" ) { print "string\nv3symlink\n"; exit 0;}
if ( $ret eq "$place.8.12.2" ) { print "gauge\n$nfsd{'v3symlink'}\n"; exit 0;}
if ( $ret eq "$place.8.13.1" ) { print "string\nv3mknod\n"; exit 0;}
if ( $ret eq "$place.8.13.2" ) { print "gauge\n$nfsd{'v3mknod'}\n"; exit 0;}
if ( $ret eq "$place.8.14.1" ) { print "string\nv3remove\n"; exit 0;}
if ( $ret eq "$place.8.14.2" ) { print "gauge\n$nfsd{'v3remove'}\n"; exit 0;}
if ( $ret eq "$place.8.15.1" ) { print "string\nv3rmdir\n"; exit 0;}
if ( $ret eq "$place.8.15.2" ) { print "gauge\n$nfsd{'v3rmdir'}\n"; exit 0;}
if ( $ret eq "$place.8.16.1" ) { print "string\nv3rename\n"; exit 0;}
if ( $ret eq "$place.8.16.2" ) { print "gauge\n$nfsd{'v3rename'}\n"; exit 0;}
if ( $ret eq "$place.8.17.1" ) { print "string\nv3link\n"; exit 0;}
if ( $ret eq "$place.8.17.2" ) { print "gauge\n$nfsd{'v3link'}\n"; exit 0;}
if ( $ret eq "$place.8.18.1" ) { print "string\nv3readdir\n"; exit 0;}
if ( $ret eq "$place.8.18.2" ) { print "gauge\n$nfsd{'v3readdir'}\n"; exit 0;}
if ( $ret eq "$place.8.19.1" ) { print "string\nv3readdirplus\n"; exit 0;}
if ( $ret eq "$place.8.19.2" ) { print "gauge\n$nfsd{'v3readdirplus'}\n"; exit 0;}
if ( $ret eq "$place.8.20.1" ) { print "string\nv3fsstat\n"; exit 0;}
if ( $ret eq "$place.8.20.2" ) { print "gauge\n$nfsd{'v3fsstat'}\n"; exit 0;}
if ( $ret eq "$place.8.21.1" ) { print "string\nv3fsinfo\n"; exit 0;}
if ( $ret eq "$place.8.21.2" ) { print "gauge\n$nfsd{'v3fsinfo'}\n"; exit 0;}
if ( $ret eq "$place.8.22.1" ) { print "string\nv3pathconf\n"; exit 0;}
if ( $ret eq "$place.8.22.2" ) { print "gauge\n$nfsd{'v3pathconf'}\n"; exit 0;}
if ( $ret eq "$place.8.23.1" ) { print "string\nv3commit\n"; exit 0;}
if ( $ret eq "$place.8.23.2" ) { print "gauge\n$nfsd{'v3commit'}\n"; exit 0;}
if ( $ret eq "$place.8.24.1" ) { print "string\nv3total\n"; exit 0;}
if ( $ret eq "$place.8.24.2" ) { print "gauge\n$nfsd{'v3total'}\n"; exit 0;}
else { print "string\nso long and thanks for all the fish\n"; exit 0; }

gathernfs.sh – utility to poll snmp and populate the RRD database

#!/bin/ksh
 
#
# Global Variables
#
 
V3CALLS=".1.3.6.1.4.1.2021.255.8"
V3TOTAL=".1.3.6.1.4.1.2021.255.8.24.2"
RRDOUT="/usr/local/web/rrd/data"
GREPOUT="proc2snmp.pl"
SNMPGET="/usr/bin/snmpget";
SNMPWALK="/usr/bin/snmpwalk";
AWK="/usr/bin/awk";
 
Usage () {
	echo "Usage: $0 -F <filername>";
	exit;
}
#
# MAIN
#
 
[ $# -lt 1 ] && Usage;
 
while [ $# -gt 0 ]
do
	case "$1" in
		-F)	FILER=$2; shift;;
		*)	break;;	
	esac
	shift
done
 
#
# get the total
#
TOTALOPS=`/usr/bin/snmpget -v2c -c public $FILER $V3TOTAL | /usr/bin/awk -F': ' '{print $NF}'`
 
#
# Do an SNMPWalk and get all the output. 
#
/usr/bin/snmpwalk -c public -v2c $FILER $V3CALLS | $AWK -F': ' '{print $2}' | grep -v $GREPOUT | while read nfsval
do
	case "$nfsval" in 
		#
		# Is the value a string or a number ?
		# strings represent the DS and the number following it is its value
		#
		*[!0-9]*) nfsval2=`echo $nfsval | awk -F'"' '{print $2}'`
					 if [ "X$DSORDER" == "X" ] 
					 then  
						DSORDER="$nfsval2" 
					 else 
						DSORDER="$DSORDER:$nfsval2"
					fi
					;;
		*) v3percent=`echo "scale=4; ($nfsval/$TOTALOPS) * 100" | bc`;
			VALUE="$VALUE:$v3percent";
			;;
	esac
done
STRING="-t $DSORDER N$VALUE";
/usr/local/bin/rrdtool update ${RRDOUT}/${FILER}.rrd $STRING
exit 0;

SNMP Configuration
Setting up snmp can be as easy as running /usr/bin/snmpconf utility, which reads the existing files and generates a new snmpd.conf after asking a set of questions. One can also manually edit the /etc/snmp/snmpd.conf file and put in the directives.

A basic snmpd.conf file is below:

[root@nfsa ~]# grep -v "#" /etc/snmp/snmpd.conf  | grep -v '^$'
pass .1.3.6.1.4.1.4413.4.1 /usr/bin/ucd5820stat
pass .1.3.6.1.4.1.2021.255 /usr/local/custom/proc2snmp.pl
syslocation  "2nd belt, 4th asteroid - beetlegeuse"
syscontact  noreply (at) blogsome (dot) com
sysservices 72
proc  nfsd 32 8
rocommunity  public

The read-only community string is “public” and we are using net-snmp pass directive to pass the control of the entire MIB tree of .1.3.6.1.4.1.2021.255 to an external program called /usr/local/custom/proc2snmp.pl. The good thing about the pass directives is that the ENTIRE subtree specified in the line is available for use by the program, which allows for future expansion – which we know always happens.

The following line in the snmpd.conf file is needed:

pass .1.3.6.1.4.1.2021.255 /usr/local/custom/proc2snmp.pl

Install the perl script in the specified location and you are ready to test!

Testing

#snmpwalk -v2c -c public localhost 1.3.6.1.4.1.2021.255
UCD-SNMP-MIB::ucdavis.255.8 = STRING: "so long and thanks for all the fish"
UCD-SNMP-MIB::ucdavis.255.8.1 = STRING: "/usr/local/custom/proc2snmp.pl"
UCD-SNMP-MIB::ucdavis.255.8.2.1 = STRING: "v3null"
UCD-SNMP-MIB::ucdavis.255.8.2.2 = Gauge32: 974
UCD-SNMP-MIB::ucdavis.255.8.3.1 = STRING: "v3getattr"
UCD-SNMP-MIB::ucdavis.255.8.3.2 = Gauge32: 17139386
UCD-SNMP-MIB::ucdavis.255.8.4.1 = STRING: "v3setattr"
UCD-SNMP-MIB::ucdavis.255.8.4.2 = Gauge32: 6848261
UCD-SNMP-MIB::ucdavis.255.8.5.1 = STRING: "v3lookup"
UCD-SNMP-MIB::ucdavis.255.8.5.2 = Gauge32: 6543679
UCD-SNMP-MIB::ucdavis.255.8.6.1 = STRING: "v3access"
UCD-SNMP-MIB::ucdavis.255.8.6.2 = Gauge32: 2918027
UCD-SNMP-MIB::ucdavis.255.8.7.1 = STRING: "v3readlink"
UCD-SNMP-MIB::ucdavis.255.8.7.2 = Gauge32: 361
UCD-SNMP-MIB::ucdavis.255.8.8.1 = STRING: "v3read"
UCD-SNMP-MIB::ucdavis.255.8.8.2 = Gauge32: 1381807
UCD-SNMP-MIB::ucdavis.255.8.9.1 = STRING: "v3create"
UCD-SNMP-MIB::ucdavis.255.8.9.2 = Gauge32: 845070
UCD-SNMP-MIB::ucdavis.255.8.10.1 = STRING: "v3mkdir"
UCD-SNMP-MIB::ucdavis.255.8.10.2 = Gauge32: 22960
UCD-SNMP-MIB::ucdavis.255.8.11.1 = STRING: "v3write"
UCD-SNMP-MIB::ucdavis.255.8.11.2 = Gauge32: 6218351
UCD-SNMP-MIB::ucdavis.255.8.12.1 = STRING: "v3symlink"
UCD-SNMP-MIB::ucdavis.255.8.12.2 = Gauge32: 185
UCD-SNMP-MIB::ucdavis.255.8.13.1 = STRING: "v3mknod"
UCD-SNMP-MIB::ucdavis.255.8.13.2 = Gauge32: 0
UCD-SNMP-MIB::ucdavis.255.8.14.1 = STRING: "v3remove"
UCD-SNMP-MIB::ucdavis.255.8.14.2 = Gauge32: 113995
UCD-SNMP-MIB::ucdavis.255.8.15.1 = STRING: "v3rmdir"
UCD-SNMP-MIB::ucdavis.255.8.15.2 = Gauge32: 2815
UCD-SNMP-MIB::ucdavis.255.8.16.1 = STRING: "v3rename"
UCD-SNMP-MIB::ucdavis.255.8.16.2 = Gauge32: 10830
UCD-SNMP-MIB::ucdavis.255.8.17.1 = STRING: "v3link"
UCD-SNMP-MIB::ucdavis.255.8.17.2 = Gauge32: 26781
UCD-SNMP-MIB::ucdavis.255.8.18.1 = STRING: "v3readdir"
UCD-SNMP-MIB::ucdavis.255.8.18.2 = Gauge32: 73914
UCD-SNMP-MIB::ucdavis.255.8.19.1 = STRING: "v3readdirplus"
UCD-SNMP-MIB::ucdavis.255.8.19.2 = Gauge32: 139878
UCD-SNMP-MIB::ucdavis.255.8.20.1 = STRING: "v3fsstat"
UCD-SNMP-MIB::ucdavis.255.8.20.2 = Gauge32: 479
UCD-SNMP-MIB::ucdavis.255.8.21.1 = STRING: "v3fsinfo"
UCD-SNMP-MIB::ucdavis.255.8.21.2 = Gauge32: 827
UCD-SNMP-MIB::ucdavis.255.8.22.1 = STRING: "v3pathconf"
UCD-SNMP-MIB::ucdavis.255.8.22.2 = Gauge32: 1
UCD-SNMP-MIB::ucdavis.255.8.23.1 = STRING: "v3commit"
UCD-SNMP-MIB::ucdavis.255.8.23.2 = Gauge32: 4491773
UCD-SNMP-MIB::ucdavis.255.8.24.1 = STRING: "v3total"
UCD-SNMP-MIB::ucdavis.255.8.24.2 = Gauge32: 46780946

Notice how I am using a subtree 1.3.6.1.4.1.2021.255.8 for nfs v3 server statistics. This way, the script can be expanded to include other status under a different OID in the same MIBOID without additional snmp reconfigurations and restarts.

Once this is setup, the next task is to collect poll the host for statistics and stick them in a database. RRDTool is a mostly commonly used tool for these purposes. RRDTool also makes generating graphs from its databases a breeze.

Setting up a RRD Database

RRDTool installation is very well covered on the RRDTool website, so there is no need to go into it again. Ben Rockwood has an excellent article titled “Getting started with RRDTool” on his website. It is a must read for anyone interested in RRDTool. Assuming you have RRD already installed , the next step is to create the database to collect all of the nfsv3 variables.

#/usr/local/bin/rrdtool create nfsd.rrd \
 DS:v3null:GAUGE:600:0:100 \
 DS:v3getattr:GAUGE:600:0:100 \
 DS:v3setattr:GAUGE:600:0:100 \
 DS:v3lookup:GAUGE:600:0:100 \
 DS:v3access:GAUGE:600:0:100 \
 DS:v3readlink:GAUGE:600:0:100 \
 DS:v3read:GAUGE:600:0:100 \
 DS:v3write:GAUGE:600:0:100 \
 DS:v3create:GAUGE:600:0:100 \
 DS:v3mkdir:GAUGE:600:0:100 \
 DS:v3symlink:GAUGE:600:0:100 \
 DS:v3mknod:GAUGE:600:0:100 \
 DS:v3remove:GAUGE:600:0:100 \
 DS:v3rmdir:GAUGE:600:0:100 \
 DS:v3rename:GAUGE:600:0:100 \
 DS:v3link:GAUGE:600:0:100 \
 DS:v3readdir:GAUGE:600:0:100 \
 DS:v3readdirplus:GAUGE:600:0:100 \
 DS:v3fsstat:GAUGE:600:0:100 \
 DS:v3fsinfo:GAUGE:600:0:100 \
 DS:v3pathconf:GAUGE:600:0:100 \
 DS:v3commit:GAUGE:600:0:100 \
 DS:v3total:GAUGE:600:0:100 \
 RRA:AVERAGE:0.5:1:600 \
 RRA:AVERAGE:0.5:24:775 \
 RRA:AVERAGE:0.5:288:797 \
 RRA:MAX:0.5:1:600 \
 RRA:MAX:0.5:6:600\
 RRA:MAX:0.5:24:775\
 RRA:MAX:0.5:288:797

This will generate a file called nfsd.rrd, which is an RRD database in the current working directory.

Populating RRD Database

Populating the rrd database is done using the rrdtool update command. I am interested in the percentages of the nfs calls so I wrote a shell script (gathernfs.sh) to poll SNMP for the absolute values and calculate the percentages. The source code is also posted at the top of this post. The script is self-explanatory and it is kinda documented. It can be installed as a cron job to run every 5 mins and update the database.

Generating graphs from the RRD Database

To verify the RRD database is getting populated, rrdtool info can be run to view the contents of the database.

#rrdtool info nfsd.rrd |more
filename = "nfsd.rrd"
rrd_version = "0003"
step = 300
last_update = 1153770616
ds[v3null].type = "GAUGE"
ds[v3null].minimal_heartbeat = 600
ds[v3null].min = 0.0000000000e+00
ds[v3null].max = 1.0000000000e+02
ds[v3null].last_ds = "UNKN"
ds[v3null].value = 0.0000000000e+00
ds[v3null].unknown_sec = 0
ds[v3getattr].type = "GAUGE"
ds[v3getattr].minimal_heartbeat = 600
ds[v3getattr].min = 0.0000000000e+00
ds[v3getattr].max = 1.0000000000e+02
ds[v3getattr].last_ds = "UNKN"
ds[v3getattr].value = 5.7567703662e+02
ds[v3getattr].unknown_sec = 0
~output truncated for brevity

Good – Let’s generate a graph displaying the percentage of getattr calls.


# rrdtool graph /usr/local/web/rrd/images/getattr.png \
 --start=end-86400 \
 --title='NFS getattr' \
 --vertical-label='% of all NFS calls' \
 --imgformat=PNG \
 --width=500 \
 --base=1000 \
 --height=120 \
 --alt-y-mrtg \
 --alt-autoscale \
 --interlaced \
 --slope-mode \
 -c BACK#000000 -c CANVAS#000000 -c GRID#bbbbbb -c MGRID#CCCCCC -c FONT#CCCCCC \
 -c FRAME#000000 -c AXIS#FFFFFF -c SHADEA#000000 -c SHADEB#000000 -c ARROW#C9B215 \
 DEF:b1=/usr/local/web/rrd/data/ibrix1.rrd:v3getattr:AVERAGE \
 DEF:b2=/usr/local/web/rrd/data/ibrix1.rrd:v3getattr:MAX \
 AREA:b1#33FFFF77:v3getattr  \
 VDEF:b1_AVERAGE=b1,AVERAGE \
 GPRINT:b1_AVERAGE:"Avg\: %8.2lf%s" \
 VDEF:b1_MAX=b1,MAXIMUM \
 GPRINT:b1_MAX:"Max\: %8.2lf%s\n" \
 LINE2:b2#00FF00

If everything went well, it is generate a rather anti-climactical output such as:

597×213

which is nothing but the dimesions of the image it just created. The resulting image looks like this:

The power of RRDTool lies not in graphing 1 measly data point such as getattr, but in graphing multiple datapoints as the first graph shows. Also, there are many front end tools that offer web-based graph generation and display. One such tool that I use extensively is drraw. Drraw allows you to generate graphing templates, dashboards to display multiple graphs on a single webpage etc.

Links
It just so happens that there are two people on the Internet, thinking the same thing and writing the same piece of code at the same time! Macdee just posted on sourceforge, a version of nfsstats.pl that does the same thing, slightly differently.

Comments

  1. Pietro Accerboni says:

    I thought to do something similar using nfsstat infos, but here you have all ‘armed and ready’: it works great and all is explained cleary.
    You save me a lot of time, good job!!

    Pietro

  2. Justin H says:

    This sounds like a great example! However, when I click on either the download or proc2snmp.pl or
    gathernfs.sh links, I get a permission denied error. Hope it’s an easy fix! :-)

  3. @Justin

    The source code is now posted on the same page. Hope you find it useful. Thx for the heads up.

Speak Your Mind

*