提高 Ubuntu 22 低延迟内核 5.15 上的网络接口中断优先级

提高 Ubuntu 22 低延迟内核 5.15 上的网络接口中断优先级

我在几台具有低延迟内核的服务器上运行 Ubuntu 18。

以前我曾经chrt提高与特定网络接口相关的中断的中断线程的优先级。

现在,在新安装的 Ubuntu 22 中,使用低延迟内核,我不再看到作为线程运行的网卡中断,因此我无法提高 chrt 的优先级。

有什么变化,有没有办法在使用 lowlatenct 内核的 Ubuntu 22 上增加网卡中断优先级?

我这样做的原因是我有几个应用程序实例,它以 10 毫秒的间隔与其他计算机同步少量数据,而这些数据一定不能延迟。

我确实找到了有关 UbuntuStudio 的信息(虽然有点旧,是 2014 年的)线程中断内核参数,并且我已经确认我的新服务器上的内核是用 构建的CONFIG_IRQ_FORCED_THREADING=y,但没有用 和CONFIG_IRQ_FORCED_THREADING_DEFAULT=y而不是CONFIG_CMDLINE_BOOL=yCONFIG_CMDLINE="threadirqs"

它是否像使用线程中断范围如果是,那么低延迟内核何时以及为何发生变化,导致无法启动线程中断默认情况下?


细节

我的服务器先前的解决方案有效运行 Ubuntu 18.04.3 LTS,内核为 4.15.0-123-lowlatency

这些是我用来提高网络接口中断优先级的步骤:

  1. 我运行grep eno1 /proc/interrupts以获取网络接口的中断列表eno1

输出:

  46:      19627  486510106 1600726600  752055110 1667993661 1758831968 1471240837  895245906  658188895  458440425  918559817 1494584384 1650824561 1492836078 1214143068  920349210  IR-PCI-MSI 5767169-edge      i40e-eno1-TxRx-0
  47:  782983498 3867361290 1350470347 1488832628 1644524179 1306935370 2202471485  885906069  266079923  769050734  767000710 1199122348 1933889985 2475859304  880359145 1207562098  IR-PCI-MSI 5767170-edge      i40e-eno1-TxRx-1
  48:  487247958  557149515 2436993704 1327150410 2327637329 1692350382 1160439225 1234994234  425076063  640500055 1555726813 1469038634 2133556615 1260984085 1076406628 1141316582  IR-PCI-MSI 5767171-edge      i40e-eno1-TxRx-2
  49: 1143028283  802247216  842065588 1507113921 1450442442 1285188073 1414989498 1386700444  961900607  949612962 1176201048  996374473 1392801596 1501055021 1244839988 1568153886  IR-PCI-MSI 5767172-edge      i40e-eno1-TxRx-3
  50:  668676162  585775185  627035622  560691459  545566733  534335154  675644369  619390835  602253370  611090233  610683285  552978104  558929823  827257425  555723069  751315724  IR-PCI-MSI 5767173-edge      i40e-eno1-TxRx-4
  51:  293791983  293018539  290310300  284911609  302624294  171231086  295128963  277943781  299153596  305718023  306559090  265176440  265877422  330072276  275946899  322208126  IR-PCI-MSI 5767174-edge      i40e-eno1-TxRx-5
  52:  191406211  164689451  184092472  163124351  197416416  147238980  150711931  166047824  174857188  176597379  163302040  163989899  169136092  206831749  154774311  195388230  IR-PCI-MSI 5767175-edge      i40e-eno1-TxRx-6
  53:  154062755  152443945  151052334  133925954  161462702  130698860  164068832   88730046  146570114  152839498  140082079  134023798  139525722  178502808  136857972  164157076  IR-PCI-MSI 5767176-edge      i40e-eno1-TxRx-7
  54:  689819152  457848749 1706899545 1350792167 1009016882  750940295 1280794513 1964042727 4293839311  604650430 1394985726 1390827662 1403478542 1032557167 1258596213 1421573353  IR-PCI-MSI 5767177-edge      i40e-eno1-TxRx-8
  55:  561179546 1569329752 1142403865 1952025068 1831162021 1786111309 1819812537 1726432246  809350013 3510981566 1252176327 1188689474 2184195381 1689746731 1808756473 1998075218  IR-PCI-MSI 5767178-edge      i40e-eno1-TxRx-9
  60:  922953009  535440685 1433001681 1370014373 1542680223 1564958146 1905615201 2245129577  606607580  751437652  653813572 1648944839 1339662143 2748851364 1328493925 2197584768  IR-PCI-MSI 5767179-edge      i40e-eno1-TxRx-10
  61:  915173255  898646194  818454435  975220367 1436058787 1258590833 1222669978 1475644158  955884853 1048337605 1028501060 1229229809 1375764724 1619404100 1174614194 1626915747  IR-PCI-MSI 5767180-edge      i40e-eno1-TxRx-11
  62:  663129689  560699700  659990638  564915959  716701151  537167244  709668792  653540652  643175469  609563694  657141242  551578294  496276894  826134713  584425798  782807812  IR-PCI-MSI 5767181-edge      i40e-eno1-TxRx-12
  63:  301178832  272290237  297349011  271058986  300930906  219283819  308344403  269722445  297793600  300198211  271567742  257075242  257586638  275439840  258617424  331559250  IR-PCI-MSI 5767182-edge      i40e-eno1-TxRx-13
  64:  161271542  160600034  162911736  135003940  159159336  126317895  145054883  142236108  153342852  158784173  149385736  130987700  145150304  179008100  100121849  168255307  IR-PCI-MSI 5767183-edge      i40e-eno1-TxRx-14
  65:  174086138  162484310  163432318  142315712  164872478  136966716  168251078  137840006  150045153  156520410  143270315  134481889  152034193  184135755  140295604  167224365  IR-PCI-MSI 5767184-edge      i40e-eno1-TxRx-15
  1. 我运行ps -el | grep irq以查找每个 irq 线程的 pideno1

输出:(我仅显示与 irqs 46-65 匹配的相关行,还有更多)

1 R     0  1255     2  1   9   - -     0 -      ?        6-01:09:56 irq/46-i40e-eno
1 S     0  1256     2  1   9   - -     0 -      ?        5-13:32:39 irq/47-i40e-eno
1 S     0  1257     2  0   9   - -     0 -      ?        5-04:09:18 irq/48-i40e-eno
1 S     0  1259     2  0   9   - -     0 -      ?        3-12:50:35 irq/49-i40e-eno
1 S     0  1261     2  0   9   - -     0 -      ?        1-19:03:44 irq/50-i40e-eno
1 S     0  1262     2  0   9   - -     0 -      ?        19:55:05 irq/51-i40e-eno
1 S     0  1263     2  0   9   - -     0 -      ?        11:48:06 irq/52-i40e-eno
1 S     0  1264     2  0   9   - -     0 -      ?        09:59:36 irq/53-i40e-eno
1 S     0  1265     2  0   9   - -     0 -      ?        5-09:23:00 irq/54-i40e-eno
1 S     0  1266     2  0   9   - -     0 -      ?        4-19:20:49 irq/55-i40e-eno
1 S     0  1267     2  0   9   - -     0 -      ?        4-19:24:25 irq/60-i40e-eno
1 S     0  1268     2  0   9   - -     0 -      ?        3-10:21:49 irq/61-i40e-eno
1 S     0  1269     2  0   9   - -     0 -      ?        1-20:29:13 irq/62-i40e-eno
1 S     0  1270     2  0   9   - -     0 -      ?        19:24:51 irq/63-i40e-eno
1 S     0  1271     2  0   9   - -     0 -      ?        10:11:11 irq/64-i40e-eno
1 S     0  1273     2  0   9   - -     0 -      ?        10:31:14 irq/65-i40e-eno
  1. 对于此网络接口的每个 irq 线程,我运行chrt -p 80 <thread-pid>,例如chrt -p 80 1255irq/46。

我实际上使用 grep、awk 和 xargs 的组合来一次性执行这三个步骤:

grep $NIC /proc/interrupts \
    | awk '{print substr($1, 0, length($1) - 1);}' \
    | xargs -I% sh -c "ps -e | grep irq/%-" \
    | awk '{print $1;}' \
    | xargs -I% sh -c "echo irq PID %; chrt -p 80 %"
me@myserver:~$ grep CONFIG_IRQ_FORCED_THREADING /boot/config-4.15.0-123-lowlatency
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_IRQ_FORCED_THREADING_DEFAULT=y

me@myserver:~$ grep CONFIG_CMDLINE /boot/config-4.15.0-123-lowlatency
CONFIG_CMDLINE_PARTITION=y
# CONFIG_CMDLINE_BOOL is not set

较新的服务器此解决方案不起作用的地方运行内核为 5.15.0-79-lowlatency 的 Ubuntu 22.04.2 LTS

前两步的结果:

  1. grep eno4 /proc/interrupts

输出:

 240:          1          0          0          0          0          0          0          0          0          0          0          0          0          0 1030184410          0  IR-PCI-MSI 5249025-edge      i40e-eno4-TxRx-0
 241:          0          0          0          0          0          0          0          0          0          0          0          0 1342657199          0          0          0  IR-PCI-MSI 5249026-edge      i40e-eno4-TxRx-1
 242:          0          0          1 1007985709          0          0          0          0          0          0          0          0          0          0          0          0  IR-PCI-MSI 5249027-edge      i40e-eno4-TxRx-2
 243:          0  995307412          0         21          0          0          0          0          0          0          0          0          0          0          0          0  IR-PCI-MSI 5249028-edge      i40e-eno4-TxRx-3
 244:          0          0          0          0         16          0 1231037185          0          0          0          0          0          0          0          0          0  IR-PCI-MSI 5249029-edge      i40e-eno4-TxRx-4
 245:          0          0          0          0  782521466          0          0          0          0          0          0          0          0          0          0          0  IR-PCI-MSI 5249030-edge      i40e-eno4-TxRx-5
 246: 1035173010          0          0          0          0          0          2          0          0          0          0          0          0          0          0          0  IR-PCI-MSI 5249031-edge      i40e-eno4-TxRx-6
 247:          0          0          0          0          0          0          0          0          0          0          0  930082293          0          0          0          0  IR-PCI-MSI 5249032-edge      i40e-eno4-TxRx-7
 248:          0          0          0          0          0          0          0  762559502          1          0          0          0          0          0          0          0  IR-PCI-MSI 5249033-edge      i40e-eno4-TxRx-8
 249:          0          0          0          0          0          0          0          0          0          0          0          0          0          0  793943748          0  IR-PCI-MSI 5249034-edge      i40e-eno4-TxRx-9
 250:          0          0          0          0          0          0          0          0          0          0         12          0  998551326          0          0          0  IR-PCI-MSI 5249035-edge      i40e-eno4-TxRx-10
 251:          0          0          0          0          0          0          0          0          0          0  975747642          1          0          0          0          0  IR-PCI-MSI 5249036-edge      i40e-eno4-TxRx-11
 252:          0 1342889081          0          0          0          0          0          0          0          0          0          0        216          0          0          0  IR-PCI-MSI 5249037-edge      i40e-eno4-TxRx-12
 253:          0          0          0          0          0          0          0          0          0          0          0          0          0         14          0  818489097  IR-PCI-MSI 5249038-edge      i40e-eno4-TxRx-13
 254:          0          0          0          0 1177974058          0          0          0          0          0          0          0          0          0          0          0  IR-PCI-MSI 5249039-edge      i40e-eno4-TxRx-14
 255:          0          0  942945250          0          0          0          0          0          0          0          0          0          0          0          0         27  IR-PCI-MSI 5249040-edge      i40e-eno4-TxRx-15
  1. ps -el | grep irq

输出:

1 S     0      14       2  0  70 -10 -     0 -      ?        00:06:33 ksoftirqd/0
1 S     0      23       2  0  70 -10 -     0 -      ?        00:14:18 ksoftirqd/1
1 S     0      29       2  0  70 -10 -     0 -      ?        00:30:44 ksoftirqd/2
1 S     0      35       2  0  70 -10 -     0 -      ?        00:03:54 ksoftirqd/3
1 S     0      41       2  0  70 -10 -     0 -      ?        00:09:49 ksoftirqd/4
1 S     0      47       2  0  70 -10 -     0 -      ?        00:02:08 ksoftirqd/5
1 S     0      53       2  0  70 -10 -     0 -      ?        00:06:32 ksoftirqd/6
1 S     0      59       2  0  70 -10 -     0 -      ?        00:05:31 ksoftirqd/7
1 S     0      65       2  0  70 -10 -     0 -      ?        00:03:34 ksoftirqd/8
1 S     0      71       2  0  70 -10 -     0 -      ?        00:00:29 ksoftirqd/9
1 S     0      77       2  0  70 -10 -     0 -      ?        00:04:11 ksoftirqd/10
1 S     0      83       2  0  70 -10 -     0 -      ?        00:02:47 ksoftirqd/11
1 S     0      89       2  0  70 -10 -     0 -      ?        00:13:01 ksoftirqd/12
1 S     0      95       2  0  70 -10 -     0 -      ?        00:01:02 ksoftirqd/13
1 S     0     101       2  0  70 -10 -     0 -      ?        00:06:02 ksoftirqd/14
1 S     0     107       2  0  70 -10 -     0 -      ?        00:06:08 ksoftirqd/15
1 I     0     197       2  0  60 -20 -     0 -      ?        00:00:00 vfio-irqfd-clea
4 S     0     924       1  0  80   0 - 20764 -      ?        00:10:46 irqbalance

内核构建参数:

me@myserver:~$ grep CONFIG_IRQ_FORCED_THREADING /boot/config-5.15.0-79-lowlatency
CONFIG_IRQ_FORCED_THREADING=y

me@myserver:~$ grep CONFIG_CMDLINE_ /boot/config-5.15.0-79-lowlatency
# CONFIG_CMDLINE_BOOL is not set
CONFIG_CMDLINE_PARTITION=y

相关内容