如何优化 10Gb 网络上 Strongswan ipsec 传输内运行的 nfs 的性能

Question

主机1：Ubuntu 18.04 主机2：Freebsd 11.2

这是我的情况...我在 10G 局域网上有 2 台主机，它们之间配置了 Strongswan IPsec 传输以保护 nfsv3。 （是的，我知道 nfsv3 很旧，我应该继续前进，但原因......）。一旦我将链中最后一个链接（主机2接口、10G交换机接口、其他10G交换机接口、主机1接口）的MTU设置为9000，我的nfs安装似乎挂起。

我相信我有两个问题需要解决。第一，一旦我的 ipsec 连接建立，通过 iperf3 测量的性能就会从 9.4Gb/秒下降到约 800Mb/秒。第二，一旦隧道启动并且所有相关接口都使用 mtu 9000，NFS 挂载就无法执行任何操作。

那么，我应该怎样做才能提高 10G LAN 上的 ipsec 性能？我的 NFS 出了什么问题？

Host1 正在使用以下 fstab 条目通过 NFS 挂载 host2：

host2:/exports/share /mnt/storage nfs    
_netdev,nofail,noatime,nolock,tcp,actimeo=1800  0 0

由于这是一个 10G 局域网，我更新了两个系统的内核选项，以便对 10G 局域网进行更多优化。

/etc/sysctl.d/10-mychanges.conf 中的 Host1 内核调整

# Maximum receive socket buffer size
net.core.rmem_max = 134217728

# Maximum send socket buffer size
net.core.wmem_max = 134217728

# Minimum, initial and max TCP Receive buffer size in Bytes
net.ipv4.tcp_rmem = 4096 87380 134217728

# Minimum, initial and max buffer space allocated
net.ipv4.tcp_wmem = 4096 65536 134217728

# Maximum number of packets queued on the input side
net.core.netdev_max_backlog = 300000

# Auto tuning
net.ipv4.tcp_moderate_rcvbuf = 1

# Don't cache ssthresh from previous connection
net.ipv4.tcp_no_metrics_save = 1

# The Hamilton TCP (HighSpeed-TCP) algorithm is a packet loss based congestion control and is more aggressive pushing up to max bandwidth (total BDP) and favors hosts with lower TTL / VARTTL.
net.ipv4.tcp_congestion_control=htcp

# If you are using jumbo frames set this to avoid MTU black holes.
net.ipv4.tcp_mtu_probing = 1

Host1 ipsec.conf

# ipsec.conf - strongSwan IPsec configuration file

# basic configuration
config setup
        charondebug="ike 4, knl 4, cfg 4"

conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        keyexchange=ikev2
        mobike=no

conn host-host

        left=192.168.0.4
        leftid=@host1
        leftcert=/etc/ipsec.d/certs/host_cert.pem
        right=192.168.0.5
        rightid=@host2
        auto=add
        authby=rsasig
        type=transport
        compress=no

主机2（自由bsd）

# $FreeBSD: releng/11.2/etc/sysctl.conf 112200 2003-03-13 18:43:50Z mux $
#
#  This file is read when going to multi-user and its contents piped thru
#  ``sysctl'' to adjust kernel values.  ``man 5 sysctl.conf'' for details.
#
# Uncomment this to prevent users from seeing information about processes that
# are being run under another UID.
#security.bsd.see_other_uids=0
# set to at least 16MB for 10GE hosts
kern.ipc.maxsockbuf=16777216

# set autotuning maximum to at least 16MB too
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.recvbuf_max=16777216

# enable send/recv autotuning
net.inet.tcp.sendbuf_auto=1
net.inet.tcp.recvbuf_auto=1

# increase autotuning step size
net.inet.tcp.sendbuf_inc=16384
net.inet.tcp.recvbuf_inc=524288

# set this on test/measurement hosts
net.inet.tcp.hostcache.expire=1

# Set congestion control algorithm to Cubic or HTCP
# Make sure the module is loaded at boot time - check loader.conf
# net.inet.tcp.cc.algorithm=cubic
net.inet.tcp.cc.algorithm=htcp

Host2 ipsec.conf

# ipsec.conf - strongSwan IPsec configuration file

# basic configuration
config setup
        charondebug="ike 4, knl 4, cfg 4"

conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        keyexchange=ikev2
        mobike=no

conn host-host

        left=192.168.0.5
        leftid=@host2
      leftcert=/usr/local/etc/ipsec.d/certs/host_cert.pem
        right=192.168.0.4
        rightid=@host1
        auto=add
        authby=rsasig
        type=transport
        compress=no