Linux移动硬盘读取大量文件时卡住的问题

Linux移动硬盘读取大量文件时卡住的问题

我有几个 USB 可移动驱动器,已连接到我的单板计算机以用作 NAS。我使用的操作系统是Armbian,这是一个类似Debian的系统。

好吧,这就是问题所在:几天前,我运行了一个名为“Syncthing”的程序(它是一个用于同步文件的工具)。我有一个包含大量文件的文件夹。我发现当“Syncthing”扫描文件时我的移动磁盘被卡住了。它只是挂在那里,没有任何提示。每次扫描都会产生这样的结果。

所以我写了一个Golang程序,同时读取大量文件。是的,卡住的情况发生了。

当卡住的情况发生时:

  1. 如果你尝试运行“df-h”,你会被卡住;
  2. 如果你尝试“cd /mnt/mydisk”,你也会被卡住;
  3. 每个从该分区读取文件的程序都会卡住,没有任何消息;

似乎如果您尝试阅读有关此可移动驱动器的任何内容,它就会冻结。我已经更改了“ulimit -n”,但它仍然不起作用。

如果我重新启动系统,一切都会很好,直到我再次尝试读取大量文件。我的三个USB移动磁盘和三个单板电脑都有同样的问题。他们有相同的系统。所以我认为这是一个系统问题。

那么我想问的是:

我怎样才能找到这个问题的原因并解决它?

以下是有关我的系统的一些信息:

# uname -a

Linux orangepizero2 4.9.255-sun50iw9 #1 SMP PREEMPT Tue Mar 1 23:28:34 UTC 2022 aarch64 GNU/Linux
# lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=sunxi-ohci/1p, 12M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=sunxi-ohci/1p, 12M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=sunxi-ohci/1p, 12M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=sunxi-ehci/1p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=sunxi-ehci/1p, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=sunxi-ehci/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 480M   (I changed usb driver to "usb-storage" before but doesn't work)

当磁盘挂起时:

# mount -l

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=481084k,nr_inodes=120271,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=98840k,mode=755)
/dev/mmcblk0p1 on / type ext4 (rw,noatime,errors=remount-ro,commit=600)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,relatime)
/dev/mmcblk0p1 on /var/log.hdd type ext4 (rw,noatime,errors=remount-ro,commit=600)
/dev/zram1 on /var/log type ext4 (rw,relatime,discard) [log2ram]
/dev/sdb1 on /mnt/mydisk type fuseblk (rw,relatime,user_id=0,group_id=0,allow_other,blksize=4096) [MiniCache]  (<- this is my removable drive)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=98836k,nr_inodes=24709,mode=700)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=98836k,nr_inodes=24709,mode=700,uid=1000,gid=1000)
# fdisk -l
...
(lot of device message)
...
Disk /dev/sdb: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: EZAZ-00GGJB0
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
Disklabel type: dos
Disk identifier: 0xa2c21b65

Device     Boot Start        End    Sectors  Size Id Type
/dev/sdb1        2048 3907026943 3907024896  1.8T  7 HPFS/NTFS/exFAT
# df -h
^C
(The command is hanging so I CTRL+C to suspend)
# strace df -h
execve("/usr/bin/df", ["df", "-h"], 0x7fde3826d8 /* 21 vars */) = 0
brk(NULL)                               = 0x55a32dc000
faccessat(AT_FDCWD, "/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=21083, ...}) = 0
mmap(NULL, 21083, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fb71e2000
close(3)                                = 0
openat(AT_FDCWD, "/lib/aarch64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0`C\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1458480, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb71e0000
mmap(NULL, 1531032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb7043000
mprotect(0x7fb71a0000, 65536, PROT_NONE) = 0
mmap(0x7fb71b0000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15d000) = 0x7fb71b0000
mmap(0x7fb71b6000, 11416, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fb71b6000
close(3)                                = 0
mprotect(0x7fb71b0000, 12288, PROT_READ) = 0
mprotect(0x55821cf000, 4096, PROT_READ) = 0
mprotect(0x7fb71ea000, 4096, PROT_READ) = 0
munmap(0x7fb71e2000, 21083)             = 0
brk(NULL)                               = 0x55a32dc000
brk(0x55a32fd000)                       = 0x55a32fd000
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=6194224, ...}) = 0
mmap(NULL, 6194224, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fb6a5a000
close(3)                                = 0
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2996, ...}) = 0
read(3, "# Locale name alias data base.\n#"..., 4096) = 2996
read(3, "", 4096)                       = 0
close(3)                                = 0
openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/proc/self/mountinfo", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "17 22 0:17 / /sys rw,nosuid,node"..., 1024) = 1024
read(3, "exec,relatime shared:11 - bpf bp"..., 1024) = 1024
read(3, "e shared:174 - tmpfs tmpfs rw,si"..., 1024) = 67
read(3, "", 1024)                       = 0
lseek(3, 0, SEEK_CUR)                   = 2115
close(3)                                = 0
ioctl(1, TCGETS, {B9600 opost isig icanon echo ...}) = 0
openat(AT_FDCWD, "/usr/lib/aarch64-linux-gnu/gconv/gconv-modules.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=27004, ...}) = 0
mmap(NULL, 27004, PROT_READ, MAP_SHARED, 3, 0) = 0x7fb6a53000
close(3)                                = 0
newfstatat(AT_FDCWD, "/dev", {st_mode=S_IFDIR|0755, st_size=3900, ...}, 0) = 0
newfstatat(AT_FDCWD, "/run", {st_mode=S_IFDIR|0755, st_size=820, ...}, 0) = 0
newfstatat(AT_FDCWD, "/", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/sys/kernel/security", {st_mode=S_IFDIR|0755, st_size=0, ...}, 0) = 0
newfstatat(AT_FDCWD, "/dev/shm", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=40, ...}, 0) = 0
newfstatat(AT_FDCWD, "/run/lock", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=80, ...}, 0) = 0
newfstatat(AT_FDCWD, "/sys/fs/cgroup", {st_mode=S_IFDIR|0555, st_size=0, ...}, 0) = 0
newfstatat(AT_FDCWD, "/sys/fs/pstore", {st_mode=S_IFDIR|0755, st_size=0, ...}, 0) = 0
newfstatat(AT_FDCWD, "/sys/fs/bpf", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=0, ...}, 0) = 0
newfstatat(AT_FDCWD, "/dev/hugepages", {st_mode=S_IFDIR|0755, st_size=0, ...}, 0) = 0
newfstatat(AT_FDCWD, "/sys/kernel/config", {st_mode=S_IFDIR|0755, st_size=0, ...}, 0) = 0
newfstatat(AT_FDCWD, "/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=240, ...}, 0) = 0
newfstatat(AT_FDCWD, "/var/log.hdd", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/var/log", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/mnt/mydisk",

(并不是我没有复制整个消息,而是我被困在这里。)

如果我遗漏了一些重要信息,请告诉我,我将编辑这篇文章。

=======[没有来自“的新消息”消息” ]=======

root@armbian:~# dmesg
...
< lot of message >
...
[   18.358530] IPVS: Creating netns size=1928 id=3
[   18.517342] proc: unrecognized mount option "hidepid=invisible" or missing value
[   19.530203] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   19.530778] yzhang..read phyaddr=0, phyid=001cc916
[   19.542162] yzhang..read phyaddr=1, phyid=001cc916
[   19.559671] libphy: gmac0: probed
[   19.595908] sunxi-gmac gmac0 eth0: eth0: Type(7) PHY ID 001cc916 at 0 IRQ poll (gmac0-0:00)
[   19.610068] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   19.689280] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[   20.014569] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[   22.649063] sunxi-gmac gmac0 eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[   22.649111] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   24.839356] IPv6: eth0: IPv6 duplicate address xxxx:xxxx:xxxx:xxxx::xxxx 
 detected!
[   25.339475] IPv6: eth0: IPv6 duplicate address xxxx:xxxx:xxxx::xxxx detected!

root@armbian:~# start_read_files
This Go program will create 1000 threads(They are called goroutine in Golang) to read the total size of all files in /mnt/mydisk.
/mnt/mydisk has 139239 files.
I print the completed count every time there is an increase of 10000.
[Count]10000
[Count]20000
[Count]30000
[Count]40000
[Count]50000
[Count]60000
[Count]70000
(stuck here, Sometimes it is 10000, sometimes it is 80000 )

root@armbian:~# df -h
^C
(CTRL+C again...Now I can be stuck here...)

root@armbian:~# dmesg | tail
[   19.542162] yzhang..read phyaddr=1, phyid=001cc916
[   19.559671] libphy: gmac0: probed
[   19.595908] sunxi-gmac gmac0 eth0: eth0: Type(7) PHY ID 001cc916 at 0 IRQ poll (gmac0-0:00)
[   19.610068] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   19.689280] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[   20.014569] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[   22.649063] sunxi-gmac gmac0 eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[   22.649111] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   24.839356] IPv6: eth0: IPv6 duplicate address xxxx:xxxx:xxxx:xxxx::xxxx detected!
[   25.339475] IPv6: eth0: IPv6 duplicate address xxxx:xxxx:xxxx::xxxx detected!
(No any new message here.)

■■■■■■■[新编辑]■■■■■■■

或许NTFS-3G里面有一些问题。经过多次测试,我找到了如何在不重新启动整个系统的情况下恢复可移动磁盘的读写。

# kill -9 <pidof ntfs-3g>
# umount /dev/sdb1
# mount /dev/sdb1 /mnt/mydisk

问题似乎变得更加困难了。这不再是“输入几个命令,问题就消失了”的麻烦......

答案1

我在这里把问题的解决方案留给和我有同样问题的人。

# ntfs-3g --version

ntfs-3g 2017.3.23AR.3 integrated FUSE 28

ntfs-3g如果可以的话更新一下。就我而言,我重新安装了最新的系统映像......

# ntfs-3g --version

ntfs-3g 2022.10.3 integrated FUSE 28

解决了!不再卡住...

相关内容