如何为 PCIe UIO 设备创建基于序列号的符号链接?

如何为 PCIe UIO 设备创建基于序列号的符号链接?

我正在开发一个基于 Linux 服务器的系统,该服务器托管多个 PCIe 数据采集板。每块板在扩展 PCIe 配置空间中都有其单独的设备序列号。这些卡作为 UIO 设备进行处理。不幸的是,似乎有时会以随机顺序枚举板。系统的正确操作需要使用与各个板永久关联的名称创建符号链接。

自然的解决方案是使用 udev (例如,基于来自http://reactivated.net/writing_udev_rules.html)。但是,udev 不提供设备属性之间的设备序列号功能。可以使用如下命令进行测试:

#udevadm info --attribute-walk /dev/uio0
[...]
  looking at device '/devices/pci0000:00/0000:00:03.0/uio/uio0':
    KERNEL=="uio0"
    SUBSYSTEM=="uio"
    DRIVER==""
    ATTR{event}=="0"
    ATTR{name}=="uio_pci_generic"
    ATTR{power/async}=="disabled"
    ATTR{power/control}=="auto"
    ATTR{power/runtime_active_kids}=="0"
    ATTR{power/runtime_active_time}=="0"
    ATTR{power/runtime_enabled}=="disabled"
    ATTR{power/runtime_status}=="unsupported"
    ATTR{power/runtime_suspended_time}=="0"
    ATTR{power/runtime_usage}=="0"
    ATTR{version}=="0.01.0"

  looking at parent device '/devices/pci0000:00/0000:00:03.0':
    KERNELS=="0000:00:03.0"
    SUBSYSTEMS=="pci"
    DRIVERS=="uio_pci_generic"
    ATTRS{ari_enabled}=="0"
    ATTRS{broken_parity_status}=="0"
    ATTRS{class}=="0x00ff00"
    ATTRS{consistent_dma_mask_bits}=="32"
    ATTRS{current_link_speed}=="Unknown"
    ATTRS{current_link_width}=="0"
    ATTRS{d3cold_allowed}=="0"
    ATTRS{device}=="0x3342"
    ATTRS{dma_mask_bits}=="32"
    ATTRS{driver_override}=="(null)"
    ATTRS{enable}=="1"
    ATTRS{irq}=="23"
    ATTRS{local_cpulist}=="0"
    ATTRS{local_cpus}=="1"
    ATTRS{max_link_speed}=="Unknown"
    ATTRS{max_link_width}=="255"
    ATTRS{msi_bus}=="1"
    ATTRS{numa_node}=="-1"
    ATTRS{power/async}=="enabled"
    ATTRS{power/control}=="on"
    ATTRS{power/runtime_active_kids}=="0"
    ATTRS{power/runtime_active_time}=="696133"
    ATTRS{power/runtime_enabled}=="forbidden"
    ATTRS{power/runtime_status}=="active"
    ATTRS{power/runtime_suspended_time}=="0"
    ATTRS{power/runtime_usage}=="2"
    ATTRS{revision}=="0x00"
    ATTRS{subsystem_device}=="0x1100"
    ATTRS{subsystem_vendor}=="0x1af4"
    ATTRS{vendor}=="0xabba"

  looking at parent device '/devices/pci0000:00':
    KERNELS=="pci0000:00"
    SUBSYSTEMS==""
    DRIVERS==""
    ATTRS{power/async}=="enabled"
    ATTRS{power/control}=="auto"
    ATTRS{power/runtime_active_kids}=="7"
    ATTRS{power/runtime_active_time}=="0"
    ATTRS{power/runtime_enabled}=="disabled"
    ATTRS{power/runtime_status}=="unsupported"
    ATTRS{power/runtime_suspended_time}=="0"
    ATTRS{power/runtime_usage}=="0"

找到udevadm关联的父 PCIe 设备,但不显示其设备序列号。

可以使用 lspci 读取 DSN:

# lspci -vvv -s 0000:00:03.0 
[...]
Capabilities: [100 v1] Device Serial Number 12-34-56-78-90-ab-cd-ef
[...]

基于此,我创建了一个工作解决方案。第一部分是存储在文件中的 udev 规则/etc/udev/rules.d/30-daq.rules

SUBSYSTEM=="uio" PROGRAM="/opt/bin/uio_namer" SYMLINK="%c"

PCIe板的DSN是通过脚本找到的,/opt/bin/uio_namer这是解决方案的第二部分:

#!/bin/sh
DEVID=abba:3342
SLOTNAME=`basename \`readlink /sys/${DEVPATH}/device\``
DLSPCI=`lspci -vvv -s ${SLOTNAME}`
#Check if it is our device
if [ -z "`echo ${DLSPCI} | grep Device | grep ${DEVID}`" ] ; then
  # It is UIO device associated with other PCIe device
  # don't create the symlink for it
  exit 1
fi
DSNCAP=`lspci -vvv -s ${SLOTNAME} | grep "Device Serial Number" `
echo daq/${DSNCAP##*" "}

该脚本还过滤掉为其他 PCIe 板创建的 UIO。对于“我们的”板,设备文件的符号链接是在 /dev/daq 目录中创建的,作为板的序列号。例如:/dev/daq/12-34-56-78-90-ab-cd-ef

该解决方案已在添加了 PCIe DAQ 板模型的 QEMU 中进行了测试。这是工作。但是,我认为这不是最佳的,以下是问题:

  1. 有什么方法可以访问 udev 规则中父设备的属性吗?
  2. 我的解决方案依赖于lspci -vvv.如果更改该格式,该解决方案将不再有效。使用“-m”选项作为“机器可读输出”没有帮助,因为它不打印 DSN。是否有任何实用程序可以以稳定的格式可靠地输出 PCIe 设备的 DSN?

答案1

我也找到了第二个问题的答案。我创建了一个显示 PCIe 设备序列号的简单应用程序。它可用于替换lspci帮助程序脚本中的调用。它还消除了输出格式的更改lspci导致先前解决方案无效的风险。来源:

/*
  Simple application displaying the PCIe Device Serial Number
  Written by Wojciech M. Zabolotny ([email protected] or [email protected])
  2021.05.23
  This code is heavily based on the examples provided in the pciutils package
  https://github.com/pciutils/pciutils therefore I release it under the same
  license: GNU GENERAL PUBLIC LICENSE Version 2, June 1991
  
  You may compile it with:
    gcc -o pcidsn pcidsn.c -lpci
*/

#include <stdio.h>
#include <pci/pci.h>
#include <string.h>
#include <stdlib.h>
#include <stdarg.h>
#include <unistd.h>

void __attribute__((noreturn))
die(char *msg, ...)
{
    va_list args;

    va_start(args, msg);
    fprintf(stderr, "pcisdn: ");
    vfprintf(stderr, msg, args);
    fputc('\n', stderr);
    exit(1);
}

int main(int argc, char * argv[])
{
    struct pci_access *pacc;
    struct pci_filter filter;
    struct pci_dev *dev;
    struct pci_cap *pdsn;
    unsigned int c;
    char namebuf[1024], *name;
    char * msg = NULL;
    pacc = pci_alloc();     /* Get the pci_access structure */
    pacc->error = die;
    pci_filter_init(pacc,&filter);
    if (msg=pci_filter_parse_slot(&filter,argv[1])) {
        die("Wrong slot definition: %s",msg);
    };
    pci_init(pacc);     /* Initialize the PCI library */

    pci_scan_bus(pacc);     /* We want to get the list of devices */
    for (dev=pacc->devices; dev; dev=dev->next) /* Iterate over all devices */
    {
        if(pci_filter_match(&filter,dev)) { /* Select only our slot */
            pdsn = pci_find_cap(dev, PCI_EXT_CAP_ID_DSN, PCI_CAP_EXTENDED);
            if(pdsn) {
                uint32_t lw = pci_read_long(dev, pdsn->addr+4);
                uint32_t hw = pci_read_long(dev, pdsn->addr+8);
                printf("%8.8lx%8.8lx\n",hw,lw);
            }
        }
    }
    pci_cleanup(pacc);      /* Close everything */
    return 0;
}

可以编译为

gcc -o pcidsn pcidsn.c -lpci

生成的应用程序应该可以通过系统路径获得。/opt/bin/uio_namer然后应将帮助程序脚本修改为:

#!/bin/sh
set -e
SLOTNAME=`basename \`readlink /sys/${DEVPATH}/device\``
DSNCAP=`pcidsn ${SLOTNAME}`
echo daq/${DSNCAP##*" "}

udev 规则/etc/udev/rules.d/30-daq.rules可能如前所述:

SUBSYSTEM=="uio"  ACTION=="add" ATTRS{vendor}=="0xabba" ATTRS{device}=="0x3342" PROGRAM="/opt/bin/uio_namer" SYMLINK="%c"

答案2

我已经找到了第一个问题的答案。在http://reactivated.net/writing_udev_rules.html有一个说法:

ATTRS - 匹配设备的 sysfs 属性,或任何父设备的 sysfs 属性

因此 PCI 供应商和设备属性的规则也可能与父设备匹配。

udev 规则/etc/udev/rules.d/30-daq.rules可以修改为:

SUBSYSTEM=="uio"  ACTION=="add" ATTRS{vendor}=="0xabba" ATTRS{device}=="0x3342" PROGRAM="/opt/bin/uio_namer" SYMLINK="%c"

辅助脚本/opt/bin/uio_namer可以简化为:

#!/bin/sh
SLOTNAME=`basename \`readlink /sys/${DEVPATH}/device\``
DSNCAP=`lspci -vvv -s ${SLOTNAME} | grep "Device Serial Number" `
echo daq/${DSNCAP##*" "}

相关内容