FreeBSD 高负载环回接口

FreeBSD 高负载环回接口

我的 FreeBSD 服务器出现了问题。

有一台FreeBSD 9.0 amd64,两块网卡em1(互联网)、em0(本地网络)配置了防火墙ipfw、natd、squid(非透明),服务器作为访问互联网的网关。

下一个问题:通过 squid 上传的速率非常低。

此刻我看到以下内容:natd,dhcpd 在通过 squid 上传时加载当时的 CPU,并且通过环回接口有大量流量。

ipfw show output

0100  655389684   36707144666 allow ip from any to any via lo0
00200          0             0 deny ip from any to 127.0.0.0/8
00300          0             0 deny ip from 127.0.0.0/8 to any
00400          0             0 deny ip from any to ::1
00500          0             0 deny ip from ::1 to any
00600          4           292 allow ipv6-icmp from :: to ff02::/16
00700          0             0 allow ipv6-icmp from fe80::/10 to fe80::/10
00800          1            76 allow ipv6-icmp from fe80::/10 to ff02::/16
00900          0             0 allow ipv6-icmp from any to any ip6 icmp6types 1
01000          0             0 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136
01100       1615         76160 deny ip from 192.168.1.1 to any in via em1
01200          0             0 deny ip from 199.69.99.11 to any in via em0
01300      46652       3705426 deny ip from any to 172.16.0.0/12 via em1
01400    3936404     345618870 deny ip from any to 192.168.0.0/16 via em1
01500          4           336 deny ip from any to 0.0.0.0/8 via em1
01600       4129        387621 deny ip from any to 169.254.0.0/16 via em1
01700          0             0 deny ip from any to 192.0.2.0/24 via em1
01800     917566      33777571 deny ip from any to 224.0.0.0/4 via em1
01900     147872      22029252 deny ip from any to 240.0.0.0/4 via em1
02000 1132194739 1190981955947 divert 8668 ip4 from any to any via em1
02100          3           248 deny ip from 172.16.0.0/12 to any via em1
02200      35925       2281289 deny ip from 192.168.0.0/16 to any via em1
02300       1808        122494 deny ip from 0.0.0.0/8 to any via em1
02400          3           174 deny ip from 169.254.0.0/16 to any via em1
02500          0             0 deny ip from 192.0.2.0/24 to any via em1
02600          0             0 deny ip from 224.0.0.0/4 to any via em1
02700          0             0 deny ip from 240.0.0.0/4 to any via em1
02800  960156249 1095316736582 allow tcp from any to any established
02900   64236062    8243196577 allow ip from any to any frag
03000         34          1756 allow tcp from any to me dst-port 25 setup
03100        193         11580 allow tcp from any to me dst-port 53 setup
03200         63          4222 allow udp from any to me dst-port 53
03300         64          8350 allow udp from me 53 to any
03400        417         24140 allow tcp from any to me dst-port 80 setup
03500        211         10472 allow ip from any to me dst-port 3389 setup
05300         77          4488 allow ip from any to me dst-port 1723 setup
05400          3           156 allow ip from any to me dst-port 8443 setup
05500       9882        590596 allow tcp from any to me dst-port 22 setup
05600          1            60 allow ip from any to me dst-port 2000 setup
05700          0             0 allow ip from any to me dst-port 2201 setup
07400    4241779     216690096 deny log logamount 1000 ip4 from any to any in via em1 setup proto tcp
07500   21135656    1048824936 allow tcp from any to any setup
07600     474447      35298081 allow udp from me to any dst-port 53 keep-state
07700        532         40612 allow udp from me to any dst-port 123 keep-state
65535 1990638432 1122305322718 allow ip from any to any

通过 squid 上传时使用 systat -ifstat

Load Average   ||| 

      Interface           Traffic               Peak                Total
           tun0  in     79.507 KB/s        232.479 KB/s           42.314 GB
                 out     2.022 MB/s          2.424 MB/s           59.662 GB

            lo0  in      4.450 MB/s          4.450 MB/s           43.723 GB
                 out     4.450 MB/s          4.450 MB/s           43.723 GB

            em1  in      2.629 MB/s          2.982 MB/s          464.533 GB
                 out     2.493 MB/s          2.875 MB/s          484.673 GB

            em0  in    240.458 KB/s        296.941 KB/s          442.368 GB
                 out   512.508 KB/s        850.857 KB/s          416.122 GB

顶部输出

PID USERNAME       THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
66885 root             1  92    0 26672K  2784K CPU3    3 528:43 65.48% natd
 9160 dhcpd            1  45    0 31032K  9280K CPU1    1   7:40 32.96% dhcpd
66455 root             1  20    0 18344K  2856K select  1 119:27  1.37% openvpn
16043 squid            1  20    0 44404K 17884K kqread  2   0:22  0.29% squid

squid配置文件

cat /usr/local/etc/squid/squid.conf
#
# Recommended minimum configuration:
#
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access deny manager

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
http_port 192.168.1.1:3128

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/squid/cache 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/squid/cache

我知道流量会经过 SQUID 好几次。但找不到原因。

答案1

我遇到了同样的问题我的解决方案是:TSO


FreeBSD 驱动程序支持 TSO(TCP 分段卸载)。您可以通过两种方式/位置启用/禁用它: - sysctl net.inet.tcp.tso=0(或 1 启用它) 执行此操作将禁用堆栈中的 TSO 并会影响所有适配器。 - ifconfig emX -tso
执行此操作将仅为此适配器禁用 TSO。要启用: - ifconfig emX tso 备注:默认情况下,只有 PCI-Express 适配器启用 TSO。其他适配器可由用户自行启用,风险自负 TSO 不支持基于 82547 和 82544 的适配器以及较旧的适配器。

禁用 TSO 后,我的 em 适配器工作正常。

相关内容