自从我的 NFS 客户端和 NFS 服务器上的 Linux 内核最近都进行了升级(Debian Wheezy 上的 3.2.78)之后,我发现客户端向服务器发送的 nfs_lookup 请求数量大幅增加(10 到 20 倍)。这导致运行在我 NFS 客户端(apache+php-fpm)上的应用程序的性能大幅下降。
什么原因导致 nfs_lookup 增长如此之快?
运行约 1 小时后,其中一个客户端的输出mountstats --nfs
如下:
Stats for ruchbah.nfs://membres mounted on /srv/membres:
NFS mount options: rw,vers=4,rsize=1048576,wsize=1048576,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.74.13,minorversion=0,local_lock=none
NFS server capabilities: caps=0x7fff,wtmult=512,dtsize=32768,bsize=0,namlen=255
NFSv4 capability flags: bm0=0xfdffbfff,bm1=0xf9be3e,acl=0x3
NFS security flavor: 1 pseudoflavor: 0
Cache events:
data cache invalidated 1231 times
attribute cache invalidated 23693734 times
inodes synced 1790 times
VFS calls:
VFS requested 46296214 inode revalidations
VFS requested 35900771 dentry revalidations
VFS called nfs_readdir() 3678 times
VFS called nfs_lookup() 35200320 times
VFS called nfs_permission() 11201 times
VFS called nfs_file_open() 22547 times
VFS called nfs_file_flush() 1400 times
VFS called nfs_lock() 7611 times
VFS called nfs_fsync() 65321 times
VFS called nfs_file_release() 3731 times
VM calls:
VFS called nfs_readpage() 856 times
VFS called nfs_readpages() 6945 times
VFS called nfs_writepage() 10789 times
VFS called nfs_writepages() 945872 times
Generic NFS counters:
File size changing operations:
truncating SETATTRs: 65272 extending WRITEs: 0
291 silly renames
short reads: 11201 short writes: 17
NFSERR_DELAYs from server: 0
NFS byte counts:
applications read 162084951 bytes via read(2)
applications wrote 19636930 bytes via write(2)
applications read 0 bytes via O_DIRECT read(2)
applications wrote 0 bytes via O_DIRECT write(2)
client read 152501261 bytes via NFS READ
client wrote 20783074 bytes via NFS WRITE
VFS 调用次数(尤其是 nfs_lookup)非常高。缓存属性失效也异常高。客户端主要使用 NFS 挂载来读取数据。