Stale NFS Mounts

If you use NFS for file sharing, the odds are you've come by some stale NFS mounts now and then.

I just had this issue on a semi-important box that I didn't want to reboot.

I'd had a directory from my laptop mounted on this box (TITAN) and turned the laptop off. Oops. I rarely turn the laptop off, so when I ssh'd in from work and attempted to cd to the laptop, all hell broke loose.

At this point, I'd tried everything - I had a few ls processes open, from trying to poke around. The mount point was symlinked to my homedir, so I couldn't even ls home without it hanging. I attempted to clean up some of the hanging processes. Interestingly, even lsof and fuser hung. Trying to umount the share gave the usual:

TITAN:/home/jantman # umount /mnt/antmanLaptop
Cannot MOUNTPROG RPC: RPC: Port mapper failure - RPC: Unable to receive
umount: /mnt/antmanLaptop: device is busy
Cannot MOUNTPROG RPC: RPC: Port mapper failure - RPC: Unable to receive
umount: /mnt/antmanLaptop: device is busy

Even more interestingly, looking at mount, I found something interesting: two exactly identical lines. As if the share was mounted twice to the same mount point.

I tried a series of umount -f and umount -r, seemingly to no avail - though umount -r did remount ONE of the instances ro.

I let the system sit for 10 minutes or so, then came back. The ro mount had disappeared, leaving me with only one mount (the way it should be).

Running umount -r again got me a single ro mount. Still, umount -f and -r were giving device busy errors. And lsof and fuser were still hanging.

At this point, even trying service nfs stop gave me:

Cannot MOUNTPROG RPC: RPC: Port mapper failure - RPC: Unable to receive
umount: /mnt/antmanLaptop: device is busy

and umount gave the same error.

I tried the advice here and actually got somewhere. After trying to kill -9 the PID of rpciod, the mount actually disappeared. And lsof worked perfectly.

So, the bottom line was trying lots of killing processes that I thought were using the mount, then trying a few umounts, stopping nfs, trying more umounts, killing rpciod, and then bingo, no more stale mount.

There has to be an easier way to do this...