我正在开发一个基于 Linux 服务器的系统,该服务器托管多个 PCIe 数据采集板。每块板在扩展 PCIe 配置空间中都有其单独的设备序列号。这些卡作为 UIO 设备进行处理。不幸的是,似乎有时会以随机顺序枚举板。系统的正确操作需要使用与各个板永久关联的名称创建符号链接。
自然的解决方案是使用 udev (例如,基于来自http://reactivated.net/writing_udev_rules.html)。但是,udev 不提供设备属性之间的设备序列号功能。可以使用如下命令进行测试:
#udevadm info --attribute-walk /dev/uio0
[...]
looking at device '/devices/pci0000:00/0000:00:03.0/uio/uio0':
KERNEL=="uio0"
SUBSYSTEM=="uio"
DRIVER==""
ATTR{event}=="0"
ATTR{name}=="uio_pci_generic"
ATTR{power/async}=="disabled"
ATTR{power/control}=="auto"
ATTR{power/runtime_active_kids}=="0"
ATTR{power/runtime_active_time}=="0"
ATTR{power/runtime_enabled}=="disabled"
ATTR{power/runtime_status}=="unsupported"
ATTR{power/runtime_suspended_time}=="0"
ATTR{power/runtime_usage}=="0"
ATTR{version}=="0.01.0"
looking at parent device '/devices/pci0000:00/0000:00:03.0':
KERNELS=="0000:00:03.0"
SUBSYSTEMS=="pci"
DRIVERS=="uio_pci_generic"
ATTRS{ari_enabled}=="0"
ATTRS{broken_parity_status}=="0"
ATTRS{class}=="0x00ff00"
ATTRS{consistent_dma_mask_bits}=="32"
ATTRS{current_link_speed}=="Unknown"
ATTRS{current_link_width}=="0"
ATTRS{d3cold_allowed}=="0"
ATTRS{device}=="0x3342"
ATTRS{dma_mask_bits}=="32"
ATTRS{driver_override}=="(null)"
ATTRS{enable}=="1"
ATTRS{irq}=="23"
ATTRS{local_cpulist}=="0"
ATTRS{local_cpus}=="1"
ATTRS{max_link_speed}=="Unknown"
ATTRS{max_link_width}=="255"
ATTRS{msi_bus}=="1"
ATTRS{numa_node}=="-1"
ATTRS{power/async}=="enabled"
ATTRS{power/control}=="on"
ATTRS{power/runtime_active_kids}=="0"
ATTRS{power/runtime_active_time}=="696133"
ATTRS{power/runtime_enabled}=="forbidden"
ATTRS{power/runtime_status}=="active"
ATTRS{power/runtime_suspended_time}=="0"
ATTRS{power/runtime_usage}=="2"
ATTRS{revision}=="0x00"
ATTRS{subsystem_device}=="0x1100"
ATTRS{subsystem_vendor}=="0x1af4"
ATTRS{vendor}=="0xabba"
looking at parent device '/devices/pci0000:00':
KERNELS=="pci0000:00"
SUBSYSTEMS==""
DRIVERS==""
ATTRS{power/async}=="enabled"
ATTRS{power/control}=="auto"
ATTRS{power/runtime_active_kids}=="7"
ATTRS{power/runtime_active_time}=="0"
ATTRS{power/runtime_enabled}=="disabled"
ATTRS{power/runtime_status}=="unsupported"
ATTRS{power/runtime_suspended_time}=="0"
ATTRS{power/runtime_usage}=="0"
找到udevadm
关联的父 PCIe 设备,但不显示其设备序列号。
可以使用 lspci 读取 DSN:
# lspci -vvv -s 0000:00:03.0
[...]
Capabilities: [100 v1] Device Serial Number 12-34-56-78-90-ab-cd-ef
[...]
基于此,我创建了一个工作解决方案。第一部分是存储在文件中的 udev 规则/etc/udev/rules.d/30-daq.rules
:
SUBSYSTEM=="uio" PROGRAM="/opt/bin/uio_namer" SYMLINK="%c"
PCIe板的DSN是通过脚本找到的,/opt/bin/uio_namer
这是解决方案的第二部分:
#!/bin/sh
DEVID=abba:3342
SLOTNAME=`basename \`readlink /sys/${DEVPATH}/device\``
DLSPCI=`lspci -vvv -s ${SLOTNAME}`
#Check if it is our device
if [ -z "`echo ${DLSPCI} | grep Device | grep ${DEVID}`" ] ; then
# It is UIO device associated with other PCIe device
# don't create the symlink for it
exit 1
fi
DSNCAP=`lspci -vvv -s ${SLOTNAME} | grep "Device Serial Number" `
echo daq/${DSNCAP##*" "}
该脚本还过滤掉为其他 PCIe 板创建的 UIO。对于“我们的”板,设备文件的符号链接是在 /dev/daq 目录中创建的,作为板的序列号。例如:/dev/daq/12-34-56-78-90-ab-cd-ef
该解决方案已在添加了 PCIe DAQ 板模型的 QEMU 中进行了测试。这是工作。但是,我认为这不是最佳的,以下是问题:
- 有什么方法可以访问 udev 规则中父设备的属性吗?
- 我的解决方案依赖于
lspci -vvv
.如果更改该格式,该解决方案将不再有效。使用“-m”选项作为“机器可读输出”没有帮助,因为它不打印 DSN。是否有任何实用程序可以以稳定的格式可靠地输出 PCIe 设备的 DSN?
答案1
我也找到了第二个问题的答案。我创建了一个显示 PCIe 设备序列号的简单应用程序。它可用于替换lspci
帮助程序脚本中的调用。它还消除了输出格式的更改lspci
导致先前解决方案无效的风险。来源:
/*
Simple application displaying the PCIe Device Serial Number
Written by Wojciech M. Zabolotny ([email protected] or [email protected])
2021.05.23
This code is heavily based on the examples provided in the pciutils package
https://github.com/pciutils/pciutils therefore I release it under the same
license: GNU GENERAL PUBLIC LICENSE Version 2, June 1991
You may compile it with:
gcc -o pcidsn pcidsn.c -lpci
*/
#include <stdio.h>
#include <pci/pci.h>
#include <string.h>
#include <stdlib.h>
#include <stdarg.h>
#include <unistd.h>
void __attribute__((noreturn))
die(char *msg, ...)
{
va_list args;
va_start(args, msg);
fprintf(stderr, "pcisdn: ");
vfprintf(stderr, msg, args);
fputc('\n', stderr);
exit(1);
}
int main(int argc, char * argv[])
{
struct pci_access *pacc;
struct pci_filter filter;
struct pci_dev *dev;
struct pci_cap *pdsn;
unsigned int c;
char namebuf[1024], *name;
char * msg = NULL;
pacc = pci_alloc(); /* Get the pci_access structure */
pacc->error = die;
pci_filter_init(pacc,&filter);
if (msg=pci_filter_parse_slot(&filter,argv[1])) {
die("Wrong slot definition: %s",msg);
};
pci_init(pacc); /* Initialize the PCI library */
pci_scan_bus(pacc); /* We want to get the list of devices */
for (dev=pacc->devices; dev; dev=dev->next) /* Iterate over all devices */
{
if(pci_filter_match(&filter,dev)) { /* Select only our slot */
pdsn = pci_find_cap(dev, PCI_EXT_CAP_ID_DSN, PCI_CAP_EXTENDED);
if(pdsn) {
uint32_t lw = pci_read_long(dev, pdsn->addr+4);
uint32_t hw = pci_read_long(dev, pdsn->addr+8);
printf("%8.8lx%8.8lx\n",hw,lw);
}
}
}
pci_cleanup(pacc); /* Close everything */
return 0;
}
可以编译为
gcc -o pcidsn pcidsn.c -lpci
生成的应用程序应该可以通过系统路径获得。/opt/bin/uio_namer
然后应将帮助程序脚本修改为:
#!/bin/sh
set -e
SLOTNAME=`basename \`readlink /sys/${DEVPATH}/device\``
DSNCAP=`pcidsn ${SLOTNAME}`
echo daq/${DSNCAP##*" "}
udev 规则/etc/udev/rules.d/30-daq.rules
可能如前所述:
SUBSYSTEM=="uio" ACTION=="add" ATTRS{vendor}=="0xabba" ATTRS{device}=="0x3342" PROGRAM="/opt/bin/uio_namer" SYMLINK="%c"
答案2
我已经找到了第一个问题的答案。在http://reactivated.net/writing_udev_rules.html有一个说法:
ATTRS - 匹配设备的 sysfs 属性,或任何父设备的 sysfs 属性
因此 PCI 供应商和设备属性的规则也可能与父设备匹配。
udev 规则/etc/udev/rules.d/30-daq.rules
可以修改为:
SUBSYSTEM=="uio" ACTION=="add" ATTRS{vendor}=="0xabba" ATTRS{device}=="0x3342" PROGRAM="/opt/bin/uio_namer" SYMLINK="%c"
辅助脚本/opt/bin/uio_namer
可以简化为:
#!/bin/sh
SLOTNAME=`basename \`readlink /sys/${DEVPATH}/device\``
DSNCAP=`lspci -vvv -s ${SLOTNAME} | grep "Device Serial Number" `
echo daq/${DSNCAP##*" "}