我有两个Linode容器.方框A是我们的通用Web服务器.它偶尔需要访问Box B,它被设置为NFS服务器.
当Box B重新启动时,无论我做什么,Box A都无法访问任何NFS共享.经过几个小时的故障排除后,我终于能够将其缩小到一步修复.
Box B重启后：
$sudo service nfs restart
这些都是CentOS 6.8盒子,是最新的.我相信,与NFS相关的软件包都是通过yum安装的.我确实在设置整个事情时遇到了一些麻烦;这不是一个平稳的过程,但在重新启动nfs服务之后,一切都很顺利.
如果我
$sudo service --status-all
发出重启之前和之后没有区别.也许这是一个时间问题？但我不知道怎么开始麻烦拍这个.我能做什么？
其他注意事项：
>我正在使用autofs自动挂载Box A中的共享,但共享也不会手动挂载
>我在Windows和Mac桌面和服务器上度过了我的日子,但我已经在Linux上运行多年的网站了.我精通我需要做的事情,但这不是我的舒适区域,我花了很多时间在谷歌上搜索如何做新事物.
我甚至不知道在哪里检查.我没有在日志中看到任何明显的东西,但告诉我要寻找什么,我会发布.
更新
在方框B：
[shorowitz@BoxB ~]$sudo chkconfig --list nfs nfs 0:off 1:off 2:on 3:on 4:on 5:on 6:off [shorowitz@BoxB ~]$sudo chkconfig --list nfslock nfslock 0:off 1:off 2:on 3:on 4:on 5:on 6:off
更新2
重新启动BoxB后,运行
$sudo showmount -e BoxB
从BoxA显示预期的挂载点,但我无法挂载它们.只需在BoxB上重启nfs即可
$sudo service nfs restart Shutting down NFS daemon: [ OK ] Shutting down NFS mountd: [ OK ] Shutting down NFS services: [ OK ] Shutting down RPC idmapd: [ OK ] FATAL: Module nfsd not found. FATAL: Error running install command for nfsd Starting NFS services: [ OK ] Starting NFS mountd: [ OK ] Starting NFS daemon: [ OK ] Starting RPC idmapd: [ OK ]
BoxA上的支架立即可用.当NFS已经工作时,这些致命的错误也出现在后续的重启中,所以我不知道它们有多相关(我以为我已经发布了它们).
其他日志信息
我在11月15日9:29发布了reboot命令
grep -i "nfs" /var/log/message* messages:Nov 15 09:29:08 BoxB kernel: nfsd: last server has exited,flushing export cache messages:Nov 15 09:29:54 BoxB kernel: RPC: Registered tcp NFSv4.1 backchannel transport module. messages:Nov 15 09:29:54 BoxB kernel: FS-Cache: Netfs 'nfs' registered for caching messages:Nov 15 09:29:54 BoxB kernel: NFS: Registering the id_resolver key type messages:Nov 15 09:29:54 BoxB kernel: nfs4filelayout_init: NFSv4 File Layout Driver Registering... messages:Nov 15 09:29:54 BoxB kernel: Installing knfsd (copyright (C) 1996 okir@monad.swb.de). messages:Nov 15 09:29:54 BoxB kernel: xenfs: not registering filesystem on non-xen platform messages:Nov 15 09:29:54 BoxB rpc.mountd[2740]: NFS v4 mounts will be disabled unless fsid=0 messages:Nov 15 09:29:54 BoxB kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory messages:Nov 15 09:29:54 BoxB kernel: NFSD: starting 90-second grace period (net ****************) messages:Nov 15 09:33:39 BoxB kernel: nfsd: last server has exited,flushing export cache messages:Nov 15 09:33:40 BoxB kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory messages:Nov 15 09:33:40 BoxB kernel: NFSD: starting 90-second grace period (net ****************)
更新3：
盒B
[shorowitz@BoxB ~]$sudo chkconfig --list | egrep "nfs|rpc"
nfs 0:off 1:off 2:on 3:on 4:on 5:on 6:off
nfslock 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rpcbind 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rpcgssd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
rpcsvcgssd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[shorowitz@BoxB ~]$sudo iptables --list -n -v
Chain INPUT (policy ACCEPT 0 packets,0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- !lo * 127.0.0.0/8 0.0.0.0/0 reject-with icmp-port-unreachable
18 710 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW icmp type 8
471 26200 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 state NEW
204K 393M ACCEPT all -- * * {BoxA IP} 0.0.0.0/0
6721 754K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
2859 168K LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 5/min burst 5 LOG flags 0 level 7 prefix `iptables_INPUT_denied: '
9229 628K REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain FORWARD (policy ACCEPT 0 packets,0 bytes)
pkts bytes target prot opt in out source destination
0 0 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 5/min burst 5 LOG flags 0 level 7 prefix `iptables_FORWARD_denied: '
0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain OUTPUT (policy ACCEPT 278K packets,8386M bytes)
pkts bytes target prot opt in out source destination
[shorowitz@BoxB ~]$sudo rpcinfo -p
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
100024 1 udp 38148 status
100024 1 tcp 45681 status
100005 1 udp 37846 mountd
100005 1 tcp 59259 mountd
100005 2 udp 59934 mountd
100005 2 tcp 42645 mountd
100005 3 udp 33867 mountd
100005 3 tcp 41823 mountd
100003 2 tcp 2049 nfs
100003 3 tcp 2049 nfs
100003 4 tcp 2049 nfs
100227 2 tcp 2049 nfs_acl
100227 3 tcp 2049 nfs_acl
100003 2 udp 2049 nfs
100003 3 udp 2049 nfs
100003 4 udp 2049 nfs
100227 2 udp 2049 nfs_acl
100227 3 udp 2049 nfs_acl
100021 1 udp 37287 nlockmgr
100021 3 udp 37287 nlockmgr
100021 4 udp 37287 nlockmgr
100021 1 tcp 37579 nlockmgr
100021 3 tcp 37579 nlockmgr
100021 4 tcp 37579 nlockmgr
这没有任何回报：
grep -v "^#" /etc/sysconfig/nfs
BoxA
$chkconfig --list | egrep "nfs|rpc"
nfs 0:off 1:off 2:on 3:on 4:on 5:on 6:off
nfslock 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rpcbind 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rpcgssd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
rpcsvcgssd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
$iptables --list -n -v
Chain INPUT (policy ACCEPT 0 packets,0 bytes)
pkts bytes target prot opt in out source destination
390K 58M ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- * * 0.0.0.0/0 127.0.0.0/8 reject-with icmp-port-unreachable
990K 7850M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
0 0 DROP all -- * * 43.255.188.145 0.0.0.0/0
8 388 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:587
11864 608K ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:25
1 40 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:636
4545 238K ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
9759 553K ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:443
24 960 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080
320 19152 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22
85 5681 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0
3254 194K LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 5/min burst 5 LOG flags 0 level 7 prefix `iptables denied: '
3634 227K DROP all -- * * 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT 0 packets,0 bytes)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 0 packets,0 bytes)
pkts bytes target prot opt in out source destination
1360K 1907M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0
$rpcinfo -p
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
100024 1 udp 55882 status
100024 1 tcp 58283 status
100011 1 udp 875 rquotad
100011 2 udp 875 rquotad
100011 1 tcp 875 rquotad
100011 2 tcp 875 rquotad
100005 1 udp 43136 mountd
100005 1 tcp 55047 mountd
100005 2 udp 51117 mountd
100005 2 tcp 42791 mountd
100005 3 udp 44511 mountd
100005 3 tcp 46535 mountd
100003 2 tcp 2049 nfs
100003 3 tcp 2049 nfs
100003 4 tcp 2049 nfs
100227 2 tcp 2049 nfs_acl
100227 3 tcp 2049 nfs_acl
100003 2 udp 2049 nfs
100003 3 udp 2049 nfs
100003 4 udp 2049 nfs
100227 2 udp 2049 nfs_acl
100227 3 udp 2049 nfs_acl
100021 1 udp 43509 nlockmgr
100021 3 udp 43509 nlockmgr
100021 4 udp 43509 nlockmgr
100021 1 tcp 38725 nlockmgr
100021 3 tcp 38725 nlockmgr
100021 4 tcp 38725 nlockmgr
$mount | grep nfs
nfsd on /proc/fs/nfsd type nfsd (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
11月14日更新
BoxA:
$cat /etc/auto.master.d/nfs
xdata -rw Boxb:/srv/nfs/xdata
xbackup -rw Boxb:/srv/nfs/xbackup
zbackups -rw Boxb:/srv/nfs/zbackups
$mount | grep nfs
mount |grep nfs
nfsd on /proc/fs/nfsd type nfsd (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
Boxb:/srv/nfs/xdata on /mnt/nfs/xdata type nfs (rw,sloppy,vers=4,addr={Boxb ip},clientaddr={Boxa ip})
你能用更多信息更新你的问题吗？
在NFS服务器上运行
在NFS服务器上运行
chkconfig --list | egrep "nfs|rpc" iptables --list -n -v rpcinfo -p
如果您的nfs服务器设置未自定义,则下一个应该不返回任何内容.
grep -v "^#" /etc/sysconfig/nfs
但是,如果您正在运行iptables – 它应该是自定义的.看这里
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/s2-nfs-nfs-firewall-config.html
在您的客户端运行
chkconfig --list | egrep "nfs|rpc" iptables --list -n -v rpcinfo -p mount | grep nfs
如果您正在运行NFSV2或NFSV3,则还需要在客户端上运行nfslock
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/s1-nfs-start.html
启动NFS时输出中看起来很奇怪 – 这是下面的行
FATAL: Module nfsd not found. FATAL: Error running install command for nfsd
我刚刚意识到一件事 – 因为你在openvz上运行 – 下面的链接适用于你的情况.看看这是否有帮助
https://openvz.org/NFS_server_inside_container
编辑1：
我今天在openvz容器上运行测试.
你看到的消息
FATAL: Module nfsd not found. FATAL: Error running install command for nfsd
是无害的.它在这里描述https://openvz.org/NFS_server_inside_container
我无法重现你的问题.重新启动nfs服务器后 – nfs客户端仍然能够浏览文件,创建新文件,从nfs共享中删除文件.
你现在可以发布自动挂载配置和输出
mount|grep nfs
来自方框A,你已经做过了.但是在卸载自动挂载的文件系统时你就做到了.所以现在在Box A cd上自动挂载dir,然后运行命令.这将提供有关安装期间使用哪些选项的信息.
此外,下次重新启动Box B并且存在此automount问题时 – 从站点A运行此命令
netstat -anp|grep ipofB
这将提供有关涉及哪些端口的信息.
在这种情况下,在B和A上收集tcpdumps也很好.
我倾向于认为你的配置没有任何问题 – 但是在vzhosts上使用iptables会发生一些奇怪的事情.不在您的容器中,而是在主机上.
您还可以尝试做的是在主机A上安装nmap,并在出现问题时 – 从A的角度扫描主机B以查看哪些端口是打开的(有些可能建议使用netstat或ss,但是这种情况下容器前面还有openvz主机防火墙,我们不知道它们是否阻塞了某些东西)
编辑2(11月25日)
你的挂载选项有点奇怪
比较你的输出
$mount | grep nfs
mount |grep nfs
nfsd on /proc/fs/nfsd type nfsd (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
Boxb:/srv/nfs/xdata on /mnt/nfs/xdata type nfs (rw,clientaddr={Boxa ip})
在下面开采.这是我的/etc/auto.misc.第6行是默认值.第16行我添加了
[root@vznfsclient /]# egrep -vn "^#|^$" /etc/auto.misc 6:cd -fstype=iso9660,ro,nosuid,nodev :/dev/cdrom 16:nfs -fstype=nfs 192.168.0.54:/nfs_export
所以当我cd到/ misc / nfs时 – 我的共享被挂载了.但请查看第12行的默认选项.
[root@vznfsclient ~]# mount|egrep -n "nfs|auto" 4:nfsd on /proc/fs/nfsd type nfsd (rw,relatime) 5:sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) 10:/etc/auto.misc on /misc type autofs (rw,relatime,fd=6,pgrp=768,timeout=300,minproto=5,maxproto=5,indirect) 11:-hosts on /net type autofs (rw,fd=12,indirect) 12:192.168.0.54:/nfs_export/ on /misc/nfs type nfs (rw,vers=3,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.54,mountvers=3,mountport=42089,mountproto=udp,local_lock=none,addr=192.168.0.54)
首先它是nfsv3,它使用的是udp.好的udp我们可以通过将/etc/auto.misc更改为此来更改为tcp
[root@vznfsclient /]# egrep -vn "^#|^$" /etc/auto.misc 6:cd -fstype=iso9660,nodev :/dev/cdrom 16:nfs -fstype=nfs,proto=tcp 192.168.0.54:/nfs_export
并且挂载选项将更改为
192.168.0.54:/nfs_export/ on /misc/nfs type nfs (rw,mountport=45378,mountproto=tcp,addr=192.168.0.54)
当我尝试在/etc/auto.misc -fstype = nfs4中使用时 – 我甚至无法cd / / misc / nfs哪种有意义,因为在容器https://openvz.org/NFS_server_inside_container内不支持openvz nfsv4
请注意,您的装载选项中包含了草率和简单的rw.这基本上意味着
>传递给mount.nfs的选项并不完全正确,所以它试图解决它.阅读man mount.nfs并搜索/ sloppy.
>我假设它正在尝试使用nfsv4.如果容器内部不支持,甚至不知道它是如何工作的.
我建议将自动挂载映射更改为正确的语法(请参阅我的示例)或参见此处的示例https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/s2-nfs-config-autofs.html
并测试.我刚刚在2个不同的openvz主机上运行的相同2个容器中运行autofs测试 – 在nfs服务器重启后 – 客户端仍然很高兴.
编辑3
我甚至无法重现你的场景.我将我的/etc/auto.misc改为下面,这几乎就是你所拥有的
[root@vznfsclient nfs]# egrep -nv "^#|^$" /etc/auto.misc 6:cd -fstype=iso9660,nodev :/dev/cdrom 16:nfs -rw 192.168.0.54:/nfs_export
重启后,cd / misc / nfs我仍然有这个
[root@vznfsclient nfs]# mount|grep nfs nfsd on /proc/fs/nfsd type nfsd (rw,relatime) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) 192.168.0.54:/nfs_export/ on /misc/nfs type nfs (rw,mountport=46453,addr=192.168.0.54)
所以在我的情况下,它正确地尝试使用nfsv3.
现在唯一要做的就是
在/etc/autofs.conf中激活debug
[root@vznfsclient nfs]# grep -i debug /etc/autofs.conf|grep -v "^#"
logging = debug
启用调试严重性以将其发送到/etc/rsyslog.conf中的/ var / log / messages.改变这个
[root@vznfsclient nfs]# grep info /etc/rsyslog.conf *.info;mail.none;authpriv.none;cron.none -/var/log/messages
对此(.info到.debug)
[root@vznfsclient nfs]# grep debug /etc/rsyslog.conf *.debug;mail.none;authpriv.none;cron.none -/var/log/messages
重新启动autofs和rsyslog,然后当你更改为自动挂载位置时 – 你应该在/ var / log / messages中看到调试输出
这是我的测试系统的输出
Nov 25 03:06:00 vznfsclient automount[583]: attempting to mount entry /misc/nfs
Nov 25 03:06:00 vznfsclient automount[583]: lookup_mount: lookup(file): looking up nfs
Nov 25 03:06:00 vznfsclient automount[583]: lookup_mount: lookup(file): nfs -> -rw#011#011#011192.168.0.54:/nfs_export
Nov 25 03:06:00 vznfsclient automount[583]: parse_mount: parse(sun): expanded entry: -rw#011#011#011192.168.0.54:/nfs_export
Nov 25 03:06:00 vznfsclient automount[583]: parse_mount: parse(sun): gathered options: rw
Nov 25 03:06:00 vznfsclient automount[583]: parse_mount: parse(sun): dequote("192.168.0.54:/nfs_export") -> 192.168.0.54:/nfs_export
Nov 25 03:06:00 vznfsclient automount[583]: parse_mount: parse(sun): core of entry: options=rw,loc=192.168.0.54:/nfs_export
Nov 25 03:06:00 vznfsclient automount[583]: sun_mount: parse(sun): mounting root /misc,mountpoint nfs,what 192.168.0.54:/nfs_export,fstype nfs,options rw
Nov 25 03:06:00 vznfsclient automount[583]: mount_mount: mount(nfs): root=/misc name=nfs what=192.168.0.54:/nfs_export,fstype=nfs,options=rw
Nov 25 03:06:00 vznfsclient automount[583]: mount_mount: mount(nfs): nfs options="rw",nobind=0,nosymlink=0,ro=0
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: called with host 192.168.0.54(192.168.0.54) proto 6 version 0x40
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: called with host 192.168.0.54(192.168.0.54) proto 6 version 0x70
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: nfs v3 rpc ping time: 0.000366
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: host 192.168.0.54 cost 365 weight 0
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: called with host 192.168.0.54(192.168.0.54) proto 17 version 0x70
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: nfs v3 rpc ping time: 0.000507
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: nfs v2 rpc ping time: 0.000692
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: host 192.168.0.54 cost 599 weight 0
Nov 25 03:06:00 vznfsclient automount[583]: prune_host_list: selected subset of hosts that support NFS3 over TCP
Nov 25 03:06:00 vznfsclient automount[583]: mount_mount: mount(nfs): calling mkdir_path /misc/nfs
Nov 25 03:06:00 vznfsclient automount[583]: mount_mount: mount(nfs): calling mount -t nfs -s -o rw 192.168.0.54:/nfs_export /misc/nfs
Nov 25 03:06:00 vznfsclient automount[583]: spawn_mount: mtab link detected,passing -n to mount
Nov 25 03:06:00 vznfsclient automount[583]: mount(nfs): mounted 192.168.0.54:/nfs_export on /misc/nfs
Nov 25 03:06:00 vznfsclient automount[583]: ioctl_send_ready: token = 28
Nov 25 03:06:00 vznfsclient automount[583]: mounted /misc/nfs