/etc/fstab
在我的系统中, systemd 启动后会创建、挂载多个逻辑卷并添加条目。
当系统第一次重新启动时,关机期间会观察到许多卸载失败的情况。
有趣的是,在后续的重启中,并没有发现这个问题。
首次重启和后续重启之间的行为差异可能是由于仅在首次重启期间,碎片*.mount
不是由 生成的systemd-fstab-generator
。我特别想修复这个问题,因为/var/log
我在这里面临数据丢失。
按照这里,我添加了覆盖配置如下systemd-journal-flush.service
:
[Unit]
PartOf=var.mount var-log.mount
[Service]
ExecStop=/usr/local/bin/debugger.sh
此外,在 ExecStop 中添加了一个调试脚本来检查是否有任何打开的 fds。
#!/bin/sh
#
date > /root/out
echo "=============" >> /root/out
fuser -m /var/log >> /root/out
echo "=============" >> /root/out
lsof | grep /var/log >> /root/out
echo "===============" >> /root/out
ps aux >> /root/out
经过此更改后,umount 在第一次重启时会失败,但在后续重启时可以正常使用。
[FAILED] Failed unmounting /var/crash/kernel.
[FAILED] Failed unmounting /var/lib/systemd/coredump.
[FAILED] Failed unmounting /var/log/audit.
[FAILED] Failed unmounting /var/sonic.
[FAILED] Failed unmounting /var/tmp.
[ OK ] Stopped target Swap.
Unmounting /mnt/sftp...
Unmounting /var/crash...
[FAILED] Failed unmounting /var/crash.
[ OK ] Stopped Flush Journal to Persistent Storage.
Unmounting /var/log...
[FAILED] Failed unmounting /var/log.
从调试脚本输出来看,lsof
和输出都是空的。我在输出中也fuser
没有发现任何可疑的东西。(状态中有一个。不确定是否需要检查。)ps
umount
D
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 27.4 0.2 188048 37548 ? Ss 15:43 5:26 /usr/lib/systemd/systemd --show-status=1 --log-target=journal-or-kmsg --log-level=notice
root 2 0.0 0.0 0 0 ? S 15:43 0:00 [kthreadd]
root 3 0.0 0.0 0 0 ? I< 15:43 0:00 [rcu_gp]
root 4 0.0 0.0 0 0 ? I< 15:43 0:00 [rcu_par_gp]
root 5 0.0 0.0 0 0 ? I 15:43 0:00 [kworker/0:0-cgroup_destroy]
root 6 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/0:0H-kblockd]
root 8 0.0 0.0 0 0 ? I< 15:43 0:00 [mm_percpu_wq]
root 9 0.0 0.0 0 0 ? S 15:43 0:00 [ksoftirqd/0]
root 10 0.2 0.0 0 0 ? I 15:43 0:02 [rcu_sched]
root 11 0.0 0.0 0 0 ? I 15:43 0:00 [rcu_bh]
root 12 0.0 0.0 0 0 ? S 15:43 0:00 [migration/0]
root 13 0.0 0.0 0 0 ? S 15:43 0:00 [cpuhp/0]
root 14 0.0 0.0 0 0 ? S 15:43 0:00 [cpuhp/1]
root 15 0.0 0.0 0 0 ? S 15:43 0:00 [migration/1]
root 16 0.0 0.0 0 0 ? S 15:43 0:00 [ksoftirqd/1]
root 18 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/1:0H-kblockd]
root 19 0.0 0.0 0 0 ? S 15:43 0:00 [cpuhp/2]
root 20 0.0 0.0 0 0 ? S 15:43 0:00 [migration/2]
root 21 0.0 0.0 0 0 ? S 15:43 0:00 [ksoftirqd/2]
root 23 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/2:0H-kblockd]
root 24 0.0 0.0 0 0 ? S 15:43 0:00 [cpuhp/3]
root 25 0.0 0.0 0 0 ? S 15:43 0:00 [migration/3]
root 26 0.0 0.0 0 0 ? S 15:43 0:00 [ksoftirqd/3]
root 28 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/3:0H-kblockd]
root 29 0.0 0.0 0 0 ? S 15:43 0:00 [cpuhp/4]
root 30 0.0 0.0 0 0 ? S 15:43 0:00 [migration/4]
root 31 0.0 0.0 0 0 ? S 15:43 0:00 [ksoftirqd/4]
root 33 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/4:0H-kblockd]
root 34 0.0 0.0 0 0 ? S 15:43 0:00 [cpuhp/5]
root 35 0.0 0.0 0 0 ? S 15:43 0:00 [migration/5]
root 36 0.0 0.0 0 0 ? S 15:43 0:00 [ksoftirqd/5]
root 38 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/5:0H-kblockd]
root 39 0.0 0.0 0 0 ? S 15:43 0:00 [cpuhp/6]
root 40 0.0 0.0 0 0 ? S 15:43 0:00 [migration/6]
root 41 0.0 0.0 0 0 ? S 15:43 0:00 [ksoftirqd/6]
root 43 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/6:0H-kblockd]
root 44 0.0 0.0 0 0 ? S 15:43 0:00 [cpuhp/7]
root 45 0.0 0.0 0 0 ? S 15:43 0:00 [migration/7]
root 46 0.0 0.0 0 0 ? S 15:43 0:00 [ksoftirqd/7]
root 48 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/7:0H-kblockd]
root 49 0.0 0.0 0 0 ? S 15:43 0:00 [kdevtmpfs]
root 50 0.0 0.0 0 0 ? I< 15:43 0:00 [netns]
root 51 0.0 0.0 0 0 ? S 15:43 0:00 [kauditd]
root 53 0.0 0.0 0 0 ? S 15:43 0:00 [khungtaskd]
root 54 0.0 0.0 0 0 ? S 15:43 0:00 [oom_reaper]
root 55 0.0 0.0 0 0 ? I< 15:43 0:00 [writeback]
root 56 0.0 0.0 0 0 ? S 15:43 0:00 [kcompactd0]
root 57 0.0 0.0 0 0 ? SN 15:43 0:00 [ksmd]
root 58 0.0 0.0 0 0 ? I< 15:43 0:00 [crypto]
root 59 0.0 0.0 0 0 ? I< 15:43 0:00 [kblockd]
root 60 0.0 0.0 0 0 ? I< 15:43 0:00 [ata_sff]
root 61 0.0 0.0 0 0 ? I< 15:43 0:00 [md]
root 62 0.0 0.0 0 0 ? I< 15:43 0:00 [devfreq_wq]
root 63 0.0 0.0 0 0 ? S 15:43 0:00 [watchdogd]
root 65 0.0 0.0 0 0 ? S 15:43 0:00 [kswapd0]
root 73 0.0 0.0 0 0 ? I< 15:43 0:00 [kthrotld]
root 74 0.0 0.0 0 0 ? S 15:43 0:00 [irq/30-pciehp]
root 76 0.0 0.0 0 0 ? I 15:43 0:00 [kworker/1:1-memcg_kmem_cache]
root 77 0.0 0.0 0 0 ? I 15:43 0:01 [kworker/3:1-events]
root 78 0.0 0.0 0 0 ? I 15:43 0:00 [kworker/4:1-cgroup_destroy]
root 79 0.0 0.0 0 0 ? I 15:43 0:00 [kworker/5:1-events]
root 80 0.0 0.0 0 0 ? I 15:43 0:01 [kworker/6:1-memcg_kmem_cache]
root 82 0.0 0.0 0 0 ? I< 15:43 0:00 [acpi_thermal_pm]
root 84 0.0 0.0 0 0 ? I< 15:43 0:00 [tpm-vtpm]
root 85 0.0 0.0 0 0 ? S 15:43 0:00 [scsi_eh_0]
root 86 0.0 0.0 0 0 ? I< 15:43 0:00 [scsi_tmf_0]
root 87 0.0 0.0 0 0 ? S 15:43 0:00 [scsi_eh_1]
root 88 0.0 0.0 0 0 ? I< 15:43 0:00 [scsi_tmf_1]
root 89 0.0 0.0 0 0 ? S 15:43 0:00 [scsi_eh_2]
root 90 0.0 0.0 0 0 ? I< 15:43 0:00 [scsi_tmf_2]
root 91 0.0 0.0 0 0 ? S 15:43 0:00 [scsi_eh_3]
root 92 0.0 0.0 0 0 ? I< 15:43 0:00 [scsi_tmf_3]
root 94 0.0 0.0 0 0 ? I 15:43 0:00 [kworker/u16:3-flush-252:18]
root 97 0.0 0.0 0 0 ? I< 15:43 0:00 [ipv6_addrconf]
root 99 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/1:1H-kblockd]
root 118 0.0 0.0 0 0 ? I< 15:43 0:00 [ixgbe]
root 126 0.0 0.0 0 0 ? I 15:43 0:00 [kworker/6:2-events]
root 133 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/2:1H-kblockd]
root 135 0.0 0.0 0 0 ? I< 15:43 0:00 [kdmflush]
root 138 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/4:1H-kblockd]
root 139 0.0 0.0 0 0 ? S 15:43 0:00 [jbd2/dm-0-8]
root 140 0.0 0.0 0 0 ? I< 15:43 0:00 [ext4-rsv-conver]
root 142 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/0:1H-kblockd]
root 145 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/6:1H-kblockd]
root 147 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/7:1H-events_highpri]
root 158 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/3:1H-kblockd]
root 166 0.2 0.3 80384 57352 ? Ss 15:43 0:02 /usr/lib/systemd/systemd-journald
root 171 0.0 0.0 0 0 ? I< 15:43 0:00 [kworker/5:1H-kblockd]
root 172 0.0 0.0 0 0 ? I< 15:43 0:00 [ipmi-msghandler]
root 173 0.0 0.0 0 0 ? I 15:43 0:00 [kworker/2:2-events_power_efficient]
root 183 0.2 0.0 7164 5260 ? Ss 15:43 0:02 /usr/lib/systemd/systemd-udevd
root 227 0.0 0.0 0 0 ? S 15:43 0:00 [scsi_eh_4]
root 228 0.0 0.0 0 0 ? I< 15:43 0:00 [scsi_tmf_4]
root 231 0.0 0.0 0 0 ? S 15:43 0:00 [usb-storage]
root 237 0.0 0.0 0 0 ? I< 15:43 0:00 [ttm_swap]
root 272 0.0 0.0 0 0 ? SN 15:43 0:00 [kipmi0]
root 68631 0.0 0.0 0 0 ? I 15:48 0:00 [kworker/3:0-cgroup_destroy]
root 68632 0.0 0.0 0 0 ? I 15:48 0:00 [kworker/1:2-mm_percpu_wq]
root 95828 0.1 0.0 0 0 ? I 15:51 0:00 [kworker/2:0-cgroup_destroy]
root 98121 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 98348 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 98522 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 98540 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 98644 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 98816 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 98879 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 98912 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 99063 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 99098 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 99132 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 99450 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 99503 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 99536 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 99856 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 100023 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 100049 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 100153 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 100189 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 100244 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 100279 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 100316 0.0 0.0 0 0 ? I< 15:51 0:00 [kdmflush]
root 101223 0.2 0.0 0 0 ? I 15:51 0:01 [kworker/u16:1-events_unbound]
root 101857 0.0 0.0 0 0 ? S 15:51 0:00 [jbd2/dm-2-8]
root 101858 0.0 0.0 0 0 ? I< 15:51 0:00 [ext4-rsv-conver]
root 101864 0.0 0.0 0 0 ? S 15:51 0:00 [jbd2/dm-3-8]
root 101865 0.0 0.0 0 0 ? I< 15:51 0:00 [ext4-rsv-conver]
root 101872 0.0 0.0 0 0 ? S 15:51 0:00 [jbd2/dm-4-8]
root 101873 0.0 0.0 0 0 ? I< 15:51 0:00 [ext4-rsv-conver]
root 101946 0.0 0.0 0 0 ? S 15:51 0:00 [jbd2/dm-14-8]
root 101947 0.0 0.0 0 0 ? I< 15:51 0:00 [ext4-rsv-conver]
root 101953 0.0 0.0 0 0 ? S 15:51 0:00 [jbd2/dm-15-8]
root 101954 0.0 0.0 0 0 ? I< 15:51 0:00 [ext4-rsv-conver]
root 101960 0.0 0.0 0 0 ? S 15:51 0:00 [jbd2/dm-16-8]
root 101961 0.0 0.0 0 0 ? I< 15:51 0:00 [ext4-rsv-conver]
root 101981 0.0 0.0 0 0 ? S 15:51 0:00 [jbd2/dm-20-8]
root 101982 0.0 0.0 0 0 ? I< 15:51 0:00 [ext4-rsv-conver]
root 101988 0.0 0.0 0 0 ? S 15:51 0:00 [jbd2/dm-22-8]
root 101989 0.0 0.0 0 0 ? I< 15:51 0:00 [ext4-rsv-conver]
root 101995 0.0 0.0 0 0 ? S 15:51 0:00 [jbd2/sda3-8]
root 101996 0.0 0.0 0 0 ? I< 15:51 0:00 [ext4-rsv-conver]
root 102071 0.1 0.0 0 0 ? I 15:51 0:00 [kworker/7:0-events]
root 102103 0.0 0.0 0 0 ? I 15:51 0:00 [kworker/5:0-rcu_gp]
root 102283 0.1 0.0 0 0 ? I 15:51 0:00 [kworker/4:0-memcg_kmem_cache]
root 102829 0.0 0.0 0 0 ? I 15:52 0:00 [kworker/7:3-cgroup_pidlist_destroy]
root 102941 0.0 0.0 0 0 ? I 15:52 0:00 [kworker/0:1-rcu_gp]
root 116853 0.0 0.0 0 0 ? I 15:59 0:00 [kworker/u16:0-events_unbound]
root 116965 0.0 0.0 0 0 ? I 15:59 0:00 [kworker/1:0-mm_percpu_wq]
root 118574 0.0 0.0 0 0 ? I 16:02 0:00 [kworker/u16:2-flush-252:22]
root 118585 0.0 0.0 7164 3708 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118586 0.0 0.0 7164 3644 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118587 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118588 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118589 0.5 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118590 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118591 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118592 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118593 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118594 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118595 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118596 0.0 0.0 7164 3620 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118597 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118598 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118599 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118600 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118601 1.6 0.0 7164 3780 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118603 0.0 0.0 7164 3640 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118604 0.0 0.0 7164 3528 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118605 0.0 0.0 7164 3528 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118606 0.0 0.0 7164 3528 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118607 0.0 0.0 7164 3528 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118608 0.0 0.0 7164 3528 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118609 0.0 0.0 7164 3528 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118610 0.0 0.0 7164 3528 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118611 0.0 0.0 7164 3528 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118612 0.0 0.0 7164 3528 ? S 16:02 0:00 /usr/lib/systemd/systemd-udevd
root 118640 0.0 0.0 0 0 ? I 16:02 0:00 [kworker/3:2-cgroup_destroy]
root 118731 0.0 0.0 0 0 ? I 16:02 0:00 [kworker/3:3-events]
root 118745 0.0 0.0 0 0 ? I 16:02 0:00 [kworker/4:2-cgroup_destroy]
root 118823 0.0 0.0 0 0 ? I 16:02 0:00 [kworker/7:1-cgroup_destroy]
root 118898 0.0 0.0 0 0 ? I 16:02 0:00 [kworker/0:2]
root 118899 0.2 0.0 0 0 ? I 16:02 0:00 [kworker/0:3-events]
root 119014 0.0 0.0 0 0 ? I 16:02 0:00 [kworker/6:0-mm_percpu_wq]
root 119052 0.0 0.0 0 0 ? I 16:02 0:00 [kworker/2:1-events]
root 119111 0.0 0.0 6136 932 ? R 16:02 0:00 /bin/umount /var/lib/docker -c
root 119117 0.0 0.0 6808 2936 ? Ss 16:02 0:00 /bin/sh /usr/local/bin/debugger.sh
root 119120 0.0 0.0 6136 876 ? D 16:02 0:00 /bin/umount /mnt/sftp -c
root 119124 0.0 0.0 6132 2036 ? R 16:02 0:00 ps aux
我的问题是:
- 我是否遗漏了运行时添加条目时所需的任何优雅处理
/etc/fstab
。有什么具体方法可以通知 systemd 创建*.mount
和systemd-fsck@*
分段? - 如果没有进程正在使用挂载点,还有什么原因会导致 umount 失败?如何进一步调试?
- 有什么方法可以解决数据丢失问题?添加另一项服务来执行延迟/强制卸载会有帮助吗?
答案1
感谢@Prem 指出卸载期间必须保持反向顺序。
可以通过将责任委托给 systemd 来解决该问题。systemd
systemd-fstab-generator
能够处理子目录的层次结构并按所需的顺序进行挂载。
从systemd-generator
手册页:
编辑 /etc/fstab 后,用户应调用 systemctl daemon-reload。这将重新运行所有生成器并导致 systemd 从磁盘重新加载单元。要实际挂载添加到 fstab 的新目录,可以使用 systemctl start /path/to/mountpoint 或 systemctl start local-fs.target。
我做的完全一样,问题解决了!