While looking at a busy CentOS 6 NFS server I decided to to write a quick script to generate a diff of the NFS v3/v4 statistics every five seconds to see what calls were being made. Easy enough to do, just read the values from the lines starting with proc(2|3|4|4ops) from /proc/net/rpc/nfsd and you are good to go. Several site provide you the mapping, but they are given in they should be in the same order as the output of ‘nfsstat -s’ if you do not want to look around.

I decided to include a diff of the output from the line ‘io … …’ as well as that will give the bytes read and written. Easy enough as well, until you start seeing negative values. This a decent system that when running full speed can do better than 10Gb/s of streaming data, but the bytes counters are down in the GBs.

Searched around a bit and all the sites confirmed that this is a just a counter that does not do any sort of ‘this is the amount changed since the last time you asked’. Decided to check the source code and caught it.


struct nfsd_stats {
...
unsigned int fh_nocache_nondir; /* filehandle not found in dcache */
unsigned int io_read; /* bytes returned to read requests */
unsigned int io_write; /* bytes passed in write requests */
unsigned int th_cnt; /* number of available threads */
...

};


Looks like it probably is a 32 bit unsigned int which would max out at 4GB. Which would explain everything. So not an accurate counter, but as long as you are not doing more than 4GB of io during your interval you can still find the diff…


if new_value > old_value
return new_value - old_value
else
return 4294967296 - old_value + new_value