我原本打算将其发布在 ServerFault 中,但我认为这可能是一个更好的地方。如果您认为有更好的地方来发布这个问题,请告诉我。
我有一个用户空间应用程序,它通过 Java NIO 的 API(也称为epoll
Linux 上)执行网络。出于演示和诊断目的,我有一个线路测试实用程序。它与 基本上是一样的iperf
。
有关环境以及测试如何运行的一些信息。
- Ubuntu 16.04 桌面今天更新(4.4.0-34-generic)
- 中断平衡已关闭
- Intel X504T1 10GbE (ixgbe) 接收器 <-> Solarflare 10GbE (sfc) 发送器
- 使用 10, 000 个 TCP 套接字
- 套接字使用操作系统默认配置
- 用户空间读缓冲区为32KB
- 读取频率不超过40hz
线路测试由单个客户端组成,该客户端通过 TCP 套接字传输尽可能多的信息。
- 每个套接字的每个 read() 允许多次调用,以获得每赫兹 98KB 的数据(32KB 缓冲区必须读取 3 次才能达到上限)
- 这意味着在 40hz 和 98KB 上限下,每个连接每秒最多可以调用 read() 120 次;共阅读3篇,840KB。
- 线路测试仪显示 read() 每秒被调用总共约 110, 000 次。
线路测试将轻松使用约 8% 使 10GbE 适配器完全饱和softirq
top - 22:04:29 up 51 min, 1 user, load average: 1.31, 1.02, 0.66
Tasks: 258 total, 1 running, 257 sleeping, 0 stopped, 0 zombie
%Cpu(s): 2.2 us, 3.6 sy, 0.0 ni, 85.6 id, 1.1 wa, 0.0 hi, 7.4 si, 0.0 st
KiB Mem : 16378912 total, 12909832 free, 2383088 used, 1085992 buff/cache
KiB Swap: 16721916 total, 16721916 free, 0 used. 13746736 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4922 jon 20 0 1553556 492552 127160 S 125.0 3.0 0:54.61 firefox
5099 jon 20 0 7212040 218396 16872 S 75.0 1.3 2:59.88 java
3194 root 20 0 722144 163812 134052 S 18.8 1.0 1:25.63 Xorg
4149 jon 20 0 1588648 147848 75344 S 6.2 0.9 0:28.63 compiz
4197 jon 20 0 544660 40600 26804 S 6.2 0.2 0:01.20 indicator-+
5186 jon 20 0 41948 3696 3084 R 6.2 0.0 0:00.01 top
1 root 20 0 119744 5884 3964 S 0.0 0.0 0:00.84 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 5:01.01 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:+
7 root 20 0 0 0 0 S 0.0 0.0 0:01.06 rcu_sched
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
10 root rt 0 0 0 0 S 0.0 0.0 0:00.04 watchdog/0
11 root rt 0 0 0 0 S 0.0 0.0 0:00.01 watchdog/1
12 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
13 root 20 0 0 0 0 S 0.0 0.0 0:08.16 ksoftirqd/1
cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 17 0 0 0 0 0 0 0 IR-IO-APIC 2-edge timer
1: 0 1 0 0 1 0 0 0 IR-IO-APIC 1-edge i8042
5: 0 0 0 0 0 0 0 0 IR-IO-APIC 5-edge parport0
8: 0 0 0 0 0 1 0 0 IR-IO-APIC 8-edge rtc0
9: 0 0 0 0 0 0 0 0 IR-IO-APIC 9-fasteoi acpi
12: 2 0 1 0 1 0 0 0 IR-IO-APIC 12-edge i8042
16: 50 6 2 6 10 0 0 3 IR-IO-APIC 16-fasteoi ehci_hcd:usb1
17: 1138 35 14 24 227 25 35 24 IR-IO-APIC 17-fasteoi snd_hda_intel
19: 0 1 0 0 0 1 0 0 IR-IO-APIC 19-fasteoi firewire_ohci
23: 11 4 10 1 7 0 0 0 IR-IO-APIC 23-fasteoi ehci_hcd:usb2
24: 0 0 0 0 0 0 0 0 DMAR-MSI 0-edge dmar0
27: 4571 1431 1142 812 1286 1442 985 730 IR-PCI-MSI 327680-edge xhci_hcd
28: 26230 3078 1744 1325 6297 2715 1703 1258 IR-PCI-MSI 512000-edge 0000:00:1f.2
29: 754 43 28 30 215 176 129 76 IR-PCI-MSI 2097152-edge eth0-rx-0
30: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2097153-edge eth0-tx-0
31: 0 0 0 0 1 0 0 0 IR-PCI-MSI 2097154-edge eth0
32: 757 64 28 33 205 169 129 66 IR-PCI-MSI 2621440-edge eth1-rx-0
33: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2621441-edge eth1-tx-0
34: 1 0 0 0 0 0 0 0 IR-PCI-MSI 2621442-edge eth1
35: 1042128 233608 58916 16705 1612687 1484813 1121118 630363 IR-PCI-MSI 1048576-edge enp2s0-TxRx-0
36: 858271 736510 372134 165262 1704892 1127381 1265752 767377 IR-PCI-MSI 1048577-edge enp2s0-TxRx-1
37: 816359 711664 426719 192686 1475309 1307882 807216 712562 IR-PCI-MSI 1048578-edge enp2s0-TxRx-2
38: 934786 714007 432100 217627 1905295 1622682 1150693 517990 IR-PCI-MSI 1048579-edge enp2s0-TxRx-3
39: 0 0 0 0 14185366 0 0 0 IR-PCI-MSI 1048580-edge enp2s0-TxRx-4
40: 0 0 0 0 0 14332864 0 0 IR-PCI-MSI 1048581-edge enp2s0-TxRx-5
41: 0 0 0 0 0 0 14617282 0 IR-PCI-MSI 1048582-edge enp2s0-TxRx-6
42: 0 0 0 0 0 0 0 14840029 IR-PCI-MSI 1048583-edge enp2s0-TxRx-7
43: 57 88 47 34 77 64 75 58 IR-PCI-MSI 1048584-edge enp2s0
44: 0 0 0 0 0 13 1 1 IR-PCI-MSI 360448-edge mei_me
45: 246 20 30 4 345 132 128 142 IR-PCI-MSI 442368-edge snd_hda_intel
46: 63933 9794 7233 4753 28843 19323 17678 11191 IR-PCI-MSI 524288-edge nvidia
NMI: 57 43 35 42 103 98 83 76 Non-maskable interrupts
LOC: 300755 258293 257168 289802 373725 262211 218677 196510 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 57 43 35 42 103 98 83 76 Performance monitoring interrupts
IWI: 0 0 0 0 1 0 0 0 IRQ work interrupts
RTR: 0 0 0 0 0 0 0 0 APIC ICR read retries
RES: 7721466 2192716 1958606 3095012 1106115 1189666 309133 169884 Rescheduling interrupts
CAL: 2598 2206 2194 1751 1976 2255 2130 2211 Function call interrupts
TLB: 5450 6659 6103 5640 4352 5128 4535 4470 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts
DFR: 0 0 0 0 0 0 0 0 Deferred Error APIC interrupts
MCE: 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 11 11 11 11 11 11 11 11 Machine check polls
ERR: 0
MIS: 0
PIN: 0 0 0 0 0 0 0 0 Posted-interrupt notification event
PIW: 0 0 0 0 0 0 0 0 Posted-interrupt wakeup event
现在,让我们对套接字读取器应用速率控制。
- 入站速率控制设置为每个连接 50KB
- 由于我们有 10, 000 个连接,因此大约为 500MB/s
- 速率控制将读取频率设置为 5hz,低于上一个示例中的 40hz。
- 速率控制的频率未对齐,这意味着并非所有连接都使用相同的起始参考,但是它们都由单个时钟控制。
- 时钟为40hz;这意味着有 40 次机会进行预定的速率控制读取。
- 在每次 5hz 速率控制读取期间,套接字最多只允许读取 10KB。因此,它每秒从套接字缓冲区读取 10KB 数据 5 次。
- 线路测试器显示 read() 每秒被调用总共约 47, 000 次。
金额softirq
从8%跃升至50-65%;中断数量几乎增加了两倍,并且有 26-5800 万个 RES 中断(每个内核),而之前为 1-700 万个。
top - 22:31:50 up 1:19, 1 user, load average: 2.30, 2.30, 1.96
Tasks: 259 total, 2 running, 257 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.3 us, 5.5 sy, 0.0 ni, 41.2 id, 0.0 wa, 0.0 hi, 50.0 si, 0.0 st
KiB Mem : 16378912 total, 11752520 free, 2189080 used, 2437312 buff/cache
KiB Swap: 16721916 total, 16721916 free, 0 used. 12590400 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3 root 20 0 0 0 0 S 82.1 0.0 26:57.43 ksoftirqd/0
5194 jon 20 0 7212040 233488 16720 S 46.2 1.4 12:08.73 java
28 root 20 0 0 0 0 S 40.2 0.0 9:04.84 ksoftirqd/4
33 root 20 0 0 0 0 S 30.9 0.0 7:26.84 ksoftirqd/5
43 root 20 0 0 0 0 R 21.6 0.0 4:26.41 ksoftirqd/7
38 root 20 0 0 0 0 S 21.3 0.0 5:37.16 ksoftirqd/6
4922 jon 20 0 1533388 475124 127784 S 5.6 2.9 2:41.82 firefox
3194 root 20 0 722448 163872 134052 S 5.3 1.0 2:50.84 Xorg
5154 jon 20 0 589896 83876 53964 S 1.7 0.5 0:26.08 plugin-con+
13 root 20 0 0 0 0 S 1.3 0.0 0:42.60 ksoftirqd/1
4548 jon 20 0 5492168 634252 43104 S 1.3 3.9 2:18.86 java
4149 jon 20 0 1604016 169732 75348 S 1.0 1.0 0:52.62 compiz
18 root 20 0 0 0 0 S 0.7 0.0 0:35.31 ksoftirqd/2
23 root 20 0 0 0 0 S 0.3 0.0 0:22.65 ksoftirqd/3
cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 17 0 0 0 0 0 0 0 IR-IO-APIC 2-edge timer
1: 0 1 0 0 1 0 0 0 IR-IO-APIC 1-edge i8042
5: 0 0 0 0 0 0 0 0 IR-IO-APIC 5-edge parport0
8: 0 0 0 0 0 1 0 0 IR-IO-APIC 8-edge rtc0
9: 0 0 0 0 0 0 0 0 IR-IO-APIC 9-fasteoi acpi
12: 2 0 1 0 1 0 0 0 IR-IO-APIC 12-edge i8042
16: 50 6 2 6 10 0 0 3 IR-IO-APIC 16-fasteoi ehci_hcd:usb1
17: 1138 35 14 24 227 25 35 24 IR-IO-APIC 17-fasteoi snd_hda_intel
19: 0 1 0 0 0 1 0 0 IR-IO-APIC 19-fasteoi firewire_ohci
23: 11 4 10 1 7 0 0 0 IR-IO-APIC 23-fasteoi ehci_hcd:usb2
24: 0 0 0 0 0 0 0 0 DMAR-MSI 0-edge dmar0
27: 6518 1966 1471 1031 4361 3847 2501 1673 IR-PCI-MSI 327680-edge xhci_hcd
28: 26732 3381 1957 1447 6687 3367 2112 1502 IR-PCI-MSI 512000-edge 0000:00:1f.2
29: 930 184 150 114 283 344 232 142 IR-PCI-MSI 2097152-edge eth0-rx-0
30: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2097153-edge eth0-tx-0
31: 0 0 0 0 1 0 0 0 IR-PCI-MSI 2097154-edge eth0
32: 899 234 138 104 277 348 236 143 IR-PCI-MSI 2621440-edge eth1-rx-0
33: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2621441-edge eth1-tx-0
34: 1 0 0 0 0 0 0 0 IR-PCI-MSI 2621442-edge eth1
35: 1339704 330929 97391 31445 2023348 1859243 1369358 782238 IR-PCI-MSI 1048576-edge enp2s0-TxRx-0
36: 1863223 3328011 1764431 788048 2411300 2677922 2540016 1742062 IR-PCI-MSI 1048577-edge enp2s0-TxRx-1
37: 1911973 3426913 2084294 955668 2216702 2894499 2008907 1723010 IR-PCI-MSI 1048578-edge enp2s0-TxRx-2
38: 2064515 3379490 2155421 1093171 2652077 3162801 2369659 1442568 IR-PCI-MSI 1048579-edge enp2s0-TxRx-3
39: 0 0 0 0 23079493 0 0 0 IR-PCI-MSI 1048580-edge enp2s0-TxRx-4
40: 0 0 0 0 0 23379687 0 0 IR-PCI-MSI 1048581-edge enp2s0-TxRx-5
41: 0 0 0 0 0 0 24721093 0 IR-PCI-MSI 1048582-edge enp2s0-TxRx-6
42: 0 0 0 0 0 0 0 25752073 IR-PCI-MSI 1048583-edge enp2s0-TxRx-7
43: 211 430 277 179 142 219 240 197 IR-PCI-MSI 1048584-edge enp2s0
44: 0 0 0 0 0 13 1 1 IR-PCI-MSI 360448-edge mei_me
45: 246 20 30 4 345 132 128 142 IR-PCI-MSI 442368-edge snd_hda_intel
46: 87961 29805 21965 14718 43334 42053 34617 23830 IR-PCI-MSI 524288-edge nvidia
NMI: 218 130 107 105 252 247 225 214 Non-maskable interrupts
LOC: 716630 636798 640606 679852 641275 555921 488433 446196 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 218 130 107 105 252 247 225 214 Performance monitoring interrupts
IWI: 0 0 0 0 3 0 0 0 IRQ work interrupts
RTR: 0 0 0 0 0 0 0 0 APIC ICR read retries
RES: 38554509 4165414 4123561 5839087 2680226 2883656 1297965 812274 Rescheduling interrupts
CAL: 3292 2356 2373 2014 2215 2496 2375 2474 Function call interrupts
TLB: 10997 21211 21364 22716 11757 23899 28023 27646 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts
DFR: 0 0 0 0 0 0 0 0 Deferred Error APIC interrupts
MCE: 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 17 17 17 17 17 17 17 17 Machine check polls
ERR: 0
MIS: 0
PIN: 0 0 0 0 0 0 0 0 Posted-interrupt notification event
PIW: 0 0 0 0 0 0 0 0 Posted-interrupt wakeup event
谁能解释为什么会发生这种情况以及如何避免这种情况?
作为参考,这里是top
使用出站速率控制 @ 500MB/s 时的情况
top - 01:26:15 up 4:13, 1 user, load average: 0.38, 0.31, 1.00
Tasks: 254 total, 1 running, 253 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1.7 us, 3.7 sy, 0.0 ni, 93.3 id, 0.1 wa, 0.0 hi, 1.2 si, 0.0 st
KiB Mem : 16378912 total, 12912528 free, 2209912 used, 1256472 buff/cache
KiB Swap: 16721916 total, 16721916 free, 0 used. 13873312 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6560 jon 20 0 7212040 204656 16836 S 38.9 1.2 0:21.37 java
3194 root 20 0 871176 206844 175404 S 1.0 1.3 12:11.62 Xorg
4149 jon 20 0 1909092 221972 99348 S 0.7 1.4 3:21.75 compiz
4548 jon 20 0 5879804 662312 45948 S 0.7 4.0 6:48.86 java
3940 jon 20 0 350840 13196 5468 S 0.3 0.1 0:20.41 ibus-daemon
4922 jon 20 0 1779380 686992 145824 S 0.3 4.2 20:38.42 firefox
5827 root 20 0 0 0 0 S 0.3 0.0 0:00.64 kworker/4:1
6341 root 20 0 0 0 0 S 0.3 0.0 0:00.93 kworker/1:2
6539 root 20 0 0 0 0 S 0.3 0.0 0:00.31 kworker/0:2
1 root 20 0 185280 5896 3964 S 0.0 0.0 0:01.01 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 107:56.20 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:+
连接 2、500 个 TCP 连接并使用速率控制可看到内部 TCP 出站数据包速率为 20K pps;跳至 5, 000 TCP 连接时,该数字跳至 105K pps;跳转到 7, 500 TCP 使出站跳转到 190K pps(这些只是确认读取的数据包 - 或者我假设)**
2:服务器上放置Solarflare卡,客户端上放置Intel X540T1;我看到 IRQ 固定ksoftirqd/0
使用 100%,并且总共si
大约12.5%
是一个核心。对于 Solarflare,RES
每个核心的中断不会超过 10, 000。**
以下是使用 Solarflare 卡时的服务器..但仅接收约 360-400MB/s,而不是目标 500MB/s
top - 11:07:55 up 16 min, 1 user, load average: 1.49, 1.09, 0.62
Tasks: 259 total, 3 running, 256 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1.5 us, 2.5 sy, 0.0 ni, 83.5 id, 0.0 wa, 0.0 hi, 12.5 si, 0.0 st
KiB Mem : 16378912 total, 12294300 free, 2356136 used, 1728476 buff/cache
KiB Swap: 16721916 total, 16721916 free, 0 used. 13067464 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3 root 20 0 0 0 0 R 99.7 0.0 5:20.82 ksoftirqd/0
4620 jon 20 0 7212040 246176 16712 S 25.6 1.5 1:24.67 java
3241 root 20 0 716936 161772 133628 R 3.3 1.0 0:15.42 Xorg
4659 jon 20 0 654928 36356 27820 S 1.0 0.2 0:00.63 gnome-term+
4103 jon 20 0 1567768 141048 75340 S 0.7 0.9 0:06.44 compiz
4542 jon 20 0 5688204 601804 43040 S 0.7 3.7 1:03.91 java
7 root 20 0 0 0 0 S 0.3 0.0 0:00.93 rcu_sched
4538 root 20 0 0 0 0 S 0.3 0.0 0:00.68 kworker/4:2
1 root 20 0 119844 5980 4028 S 0.0 0.0 0:00.84 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:+
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
10 root rt 0 0 0 0 S 0.0 0.0 0:00.02 watchdog/0
11 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
12 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
13 root 20 0 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/1
cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 17 0 0 0 0 0 0 0 IR-IO-APIC 2-edge timer
1: 1 0 0 1 0 0 0 0 IR-IO-APIC 1-edge i8042
5: 0 0 0 0 0 0 0 0 IR-IO-APIC 5-edge parport0
8: 0 0 0 0 0 1 0 0 IR-IO-APIC 8-edge rtc0
9: 0 0 0 0 0 0 0 0 IR-IO-APIC 9-fasteoi acpi
12: 1 0 1 0 1 0 1 0 IR-IO-APIC 12-edge i8042
16: 61 2 1 3 7 2 1 0 IR-IO-APIC 16-fasteoi ehci_hcd:usb1
17: 1166 55 10 19 245 45 13 19 IR-IO-APIC 17-fasteoi snd_hda_intel
19: 0 0 0 0 2 0 0 0 IR-IO-APIC 19-fasteoi firewire_ohci
23: 26 1 2 0 1 2 0 1 IR-IO-APIC 23-fasteoi ehci_hcd:usb2
24: 0 0 0 0 0 0 0 0 DMAR-MSI 0-edge dmar0
27: 1723 170 168 126 1603 166 135 47 IR-PCI-MSI 327680-edge xhci_hcd
28: 24980 1714 933 754 7492 1546 1202 936 IR-PCI-MSI 512000-edge 0000:00:1f.2
29: 298 2 1 7 159 4 6 1 IR-PCI-MSI 2097152-edge eth0-rx-0
30: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2097153-edge eth0-tx-0
31: 1 0 0 0 0 0 0 0 IR-PCI-MSI 2097154-edge eth0
32: 16878 5179 2952 3044 18575 7842 3822 3939 IR-PCI-MSI 1048576-edge enp2s0f0-0
33: 16174 4967 2787 2583 19305 7883 3507 3862 IR-PCI-MSI 1048577-edge enp2s0f0-1
34: 16707 5192 2952 2659 18031 8588 3496 4393 IR-PCI-MSI 1048578-edge enp2s0f0-2
35: 17726 5431 2951 2746 17183 8105 3529 4238 IR-PCI-MSI 1048579-edge enp2s0f0-3
36: 6 1 0 3 6 3 0 1 IR-PCI-MSI 1050624-edge enp2s0f1-0
37: 1 1 0 0 0 0 0 0 IR-PCI-MSI 1050625-edge enp2s0f1-1
38: 1 1 0 0 0 0 0 0 IR-PCI-MSI 1050626-edge enp2s0f1-2
39: 1 1 0 0 0 0 0 0 IR-PCI-MSI 1050627-edge enp2s0f1-3
40: 414 12 9 3 0 14 18 8 IR-PCI-MSI 2621440-edge eth1-rx-0
41: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2621441-edge eth1-tx-0
42: 1 0 0 0 0 0 0 0 IR-PCI-MSI 2621442-edge eth1
43: 0 0 0 0 10 0 5 0 IR-PCI-MSI 360448-edge mei_me
44: 95 26 8 33 398 384 51 16 IR-PCI-MSI 442368-edge snd_hda_intel
45: 17400 1413 1135 806 17781 1714 1401 988 IR-PCI-MSI 524288-edge nvidia
NMI: 37 3 5 3 2 1 1 1 Non-maskable interrupts
LOC: 112894 53399 87350 46718 43552 19663 25436 19705 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 37 3 5 3 2 1 1 1 Performance monitoring interrupts
IWI: 0 0 0 0 0 0 0 0 IRQ work interrupts
RTR: 0 0 0 0 0 0 0 0 APIC ICR read retries
RES: 1808 7668 9364 1244 4161 2554 9171 954 Rescheduling interrupts
CAL: 1900 2028 1497 1984 1862 1931 2118 2004 Function call interrupts
TLB: 1991 2539 3176 2985 3176 2458 1612 2087 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts
DFR: 0 0 0 0 0 0 0 0 Deferred Error APIC interrupts
MCE: 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 5 5 5 5 5 5 5 5 Machine check polls
ERR: 0
MIS: 0
PIN: 0 0 0 0 0 0 0 0 Posted-interrupt notification event
PIW: 0 0 0 0 0 0 0 0 Posted-interrupt wakeup event
答案1
问题最终是使用默认配置的套接字进行速率控制,导致内部 TCP 缓冲区大小由于读取时间缓慢而自动调整为越来越大的情况。 (默认最大大小约为 6MB)当大小自动增长时,TCP 紧凑进程将开始疯狂地搅动,从而吞噬所有软中断。解决此问题的方法是在使用速率控制时设置显式 TCP 缓冲区大小以防止这种异常行为。