Systemd 服务每 30 分钟自发重启一次

Systemd 服务每 30 分钟自发重启一次

一些服务正在重新启动,但我无法确定原因。 Ubuntu 20.04.4 服务器正在运行 Nexus,它陷入了重启的狂潮中。它不是一个新的虚拟机,已经在 Ubuntu 20.04.4 上运行了一段时间,并于上周应用了补丁。它在 AWS 上运行。

大约 8 个月前,我在设置运行 Atlassian Bamboo 的新内部 VMWare Ubuntu 20.04.x 后遇到了同样的问题。除了将 Nexus 服务替换为 Bamboo 服务之外,与下面的日志输出几乎相同。我无法确定根本原因,因此我建立了一个新的虚拟机并重新开始。它还没有生产,所以我有这个选择。

此时,我觉得我需要确定根本原因和解决方案,因为它可能是一个重复出现的问题。

我无法确定原因或继续进行故障排除步骤。

这是一个小时的日志片段,但几个小时内几乎是一样的。有人以前遇到过这种情况或者可以提供有关如何调查此问题的建议吗?

Aug  2 10:54:01 beta-nexus dbus-daemon[585]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.1989' (uid=0 pid=362880 comm="/usr/bin/hostnamectl " label="unconfined")
Aug  2 10:54:01 beta-nexus systemd[1]: Starting Hostname Service...
Aug  2 10:54:02 beta-nexus dbus-daemon[585]: [system] Successfully activated service 'org.freedesktop.hostname1'
Aug  2 10:54:02 beta-nexus systemd[1]: Started Hostname Service.
Aug  2 10:54:06 beta-nexus dbus-daemon[585]: [system] Activating via systemd: service name='org.freedesktop.PackageKit' unit='packagekit.service' requested by ':1.1991' (uid=0 pid=363670 comm="/usr/bin/gdbus call --system --dest org.freedeskto" label="unconfined")
Aug  2 10:54:06 beta-nexus systemd[1]: Starting PackageKit Daemon...
Aug  2 10:54:06 beta-nexus PackageKit: daemon start
Aug  2 10:54:06 beta-nexus dbus-daemon[585]: [system] Successfully activated service 'org.freedesktop.PackageKit'
Aug  2 10:54:06 beta-nexus systemd[1]: Started PackageKit Daemon.
Aug  2 10:54:11 beta-nexus systemd[1]: Stopping Nexus Service...
Aug  2 10:54:11 beta-nexus nexus[363835]: Shutting down nexus
Aug  2 10:54:14 beta-nexus nexus[363835]: Stopped.
Aug  2 10:54:14 beta-nexus systemd[1]: nexus.service: Succeeded.
Aug  2 10:54:14 beta-nexus systemd[1]: Stopped Nexus Service.
Aug  2 10:54:16 beta-nexus systemd[1]: Starting Nexus Service...
Aug  2 10:54:16 beta-nexus nexus[364093]: Starting nexus
Aug  2 10:54:16 beta-nexus systemd[1]: Started Nexus Service.
Aug  2 10:54:32 beta-nexus systemd[1]: systemd-hostnamed.service: Succeeded.
Aug  2 10:59:12 beta-nexus PackageKit: daemon quit
Aug  2 10:59:12 beta-nexus systemd[1]: packagekit.service: Succeeded.
Aug  2 11:17:01 beta-nexus CRON[364689]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Aug  2 11:24:22 beta-nexus dbus-daemon[585]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.2000' (uid=0 pid=364954 comm="/usr/bin/hostnamectl " label="unconfined")
Aug  2 11:24:22 beta-nexus systemd[1]: Starting Hostname Service...
Aug  2 11:24:22 beta-nexus dbus-daemon[585]: [system] Successfully activated service 'org.freedesktop.hostname1'
Aug  2 11:24:22 beta-nexus systemd[1]: Started Hostname Service.
Aug  2 11:24:27 beta-nexus dbus-daemon[585]: [system] Activating via systemd: service name='org.freedesktop.PackageKit' unit='packagekit.service' requested by ':1.2002' (uid=0 pid=365742 comm="/usr/bin/gdbus call --system --dest org.freedeskto" label="unconfined")
Aug  2 11:24:27 beta-nexus systemd[1]: Starting PackageKit Daemon...
Aug  2 11:24:27 beta-nexus PackageKit: daemon start
Aug  2 11:24:27 beta-nexus dbus-daemon[585]: [system] Successfully activated service 'org.freedesktop.PackageKit'
Aug  2 11:24:27 beta-nexus systemd[1]: Started PackageKit Daemon.
Aug  2 11:24:31 beta-nexus systemd[1]: Stopping Nexus Service...
Aug  2 11:24:31 beta-nexus nexus[365908]: Shutting down nexus
Aug  2 11:24:34 beta-nexus nexus[365908]: Stopped.
Aug  2 11:24:34 beta-nexus systemd[1]: nexus.service: Succeeded.
Aug  2 11:24:34 beta-nexus systemd[1]: Stopped Nexus Service.
Aug  2 11:24:36 beta-nexus systemd[1]: Starting Nexus Service...
Aug  2 11:24:36 beta-nexus nexus[366166]: Starting nexus
Aug  2 11:24:36 beta-nexus systemd[1]: Started Nexus Service.
Aug  2 11:24:53 beta-nexus systemd[1]: systemd-hostnamed.service: Succeeded.
Aug  2 11:29:32 beta-nexus PackageKit: daemon quit
Aug  2 11:29:32 beta-nexus systemd[1]: packagekit.service: Succeeded.
Aug  2 11:31:17 beta-nexus snapd[601]: storehelpers.go:722: cannot refresh: snap has no updates available: "amazon-ssm-agent", "core", "core18", "core20"
Aug  2 11:31:17 beta-nexus snapd[601]: autorefresh.go:539: auto-refresh: all snaps are up-to-date
Aug  2 11:39:45 beta-nexus systemd[1]: Starting Message of the Day...
Aug  2 11:39:46 beta-nexus 50-motd-news[366751]:  * Ubuntu ....
Aug  2 11:39:46 beta-nexus systemd[1]: motd-news.service: Succeeded.
Aug  2 11:39:46 beta-nexus systemd[1]: Finished Message of the Day.
Aug  2 11:54:44 beta-nexus dbus-daemon[585]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.2011' (uid=0 pid=367055 comm="/usr/bin/hostnamectl " label="unconfined")
Aug  2 11:54:44 beta-nexus systemd[1]: Starting Hostname Service...
Aug  2 11:54:44 beta-nexus dbus-daemon[585]: [system] Successfully activated service 'org.freedesktop.hostname1'
Aug  2 11:54:44 beta-nexus systemd[1]: Started Hostname Service.
Aug  2 11:54:48 beta-nexus dbus-daemon[585]: [system] Activating via systemd: service name='org.freedesktop.PackageKit' unit='packagekit.service' requested by ':1.2013' (uid=0 pid=367843 comm="/usr/bin/gdbus call --system --dest org.freedeskto" label="unconfined")
Aug  2 11:54:48 beta-nexus systemd[1]: Starting PackageKit Daemon...
Aug  2 11:54:48 beta-nexus PackageKit: daemon start
Aug  2 11:54:48 beta-nexus dbus-daemon[585]: [system] Successfully activated service 'org.freedesktop.PackageKit'
Aug  2 11:54:48 beta-nexus systemd[1]: Started PackageKit Daemon.
Aug  2 11:54:55 beta-nexus systemd[1]: Stopping Nexus Service...
Aug  2 11:54:55 beta-nexus nexus[368017]: Shutting down nexus
Aug  2 11:54:58 beta-nexus nexus[368017]: Stopped.
Aug  2 11:54:58 beta-nexus systemd[1]: nexus.service: Succeeded.
Aug  2 11:54:58 beta-nexus systemd[1]: Stopped Nexus Service.
Aug  2 11:55:00 beta-nexus systemd[1]: Starting Nexus Service...
Aug  2 11:55:00 beta-nexus nexus[368276]: Starting nexus
Aug  2 11:55:00 beta-nexus systemd[1]: Started Nexus Service.
Aug  2 11:55:14 beta-nexus systemd[1]: systemd-hostnamed.service: Succeeded.

这篇文章最初发布在 stackoverflow 中,但有一个友好的建议将其发布在这里。

编辑:添加了 nexus.service 文件。我已经确认它与在 Ubuntu 20.04.4 上运行的其他两台没有此问题的 Nexus 服务器相同。

[Unit]
Description = Nexus Service
After = network.target

[Service]
Type = forking
LimitNOFILE = 65536
WorkingDirectory = /opt/sonatype/nexus
ExecStart = /opt/sonatype/nexus/bin/nexus start
ExecStop = /opt/sonatype/nexus/bin/nexus stop
User = nexus
Group = nexus
Restart = on-abort

[Install]
WantedBy = multi-user.target

编辑:其他信息可能有帮助,也可能没有帮助...我有三台 Nexus 服务器。全部由厨师维护。所有 Ubuntu 20.04.4。一个在内部 VMWare 上,两个在 AWS 上。 VMWare 和其他 AWS 设置运行正常。另外,我需要纠正自己,它最近没有升级到 Ubuntu 20.04.4,而是上周与其他版本一起修补了。

编辑:添加了列出的计时器systemctl list-timers

NEXT                        LEFT          LAST                        PASSED       UNIT                         ACTIVATES                     
Wed 2022-08-03 06:59:48 EDT 1min 4s left  Tue 2022-08-02 06:51:03 EDT 24h ago      apt-daily-upgrade.timer      apt-daily-upgrade.service     
Wed 2022-08-03 07:10:24 EDT 11min left    Tue 2022-08-02 20:07:47 EDT 10h ago      fwupd-refresh.timer          fwupd-refresh.service         
Wed 2022-08-03 10:19:46 EDT 3h 21min left Wed 2022-08-03 04:04:26 EDT 2h 54min ago ua-timer.timer               ua-timer.service              
Wed 2022-08-03 14:12:57 EDT 7h left       Tue 2022-08-02 14:12:57 EDT 16h ago      systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Wed 2022-08-03 16:48:43 EDT 9h left       Tue 2022-08-02 18:26:57 EDT 12h ago      apt-daily.timer              apt-daily.service             
Wed 2022-08-03 21:01:28 EDT 14h left      Wed 2022-08-03 04:30:02 EDT 2h 28min ago motd-news.timer              motd-news.service             
Thu 2022-08-04 00:00:00 EDT 17h left      Wed 2022-08-03 00:00:37 EDT 6h ago       logrotate.timer              logrotate.service             
Thu 2022-08-04 00:00:00 EDT 17h left      Wed 2022-08-03 00:00:37 EDT 6h ago       man-db.timer                 man-db.service                
Sun 2022-08-07 03:10:37 EDT 3 days left   Sun 2022-07-31 03:10:29 EDT 3 days ago   e2scrub_all.timer            e2scrub_all.service           
Mon 2022-08-08 00:00:00 EDT 4 days left   Mon 2022-08-01 00:00:47 EDT 2 days ago   fstrim.timer                 fstrim.service 

编辑:添加了 systemd 日志片段。多米诺骨牌以相同的条目开始:

Aug 03 06:36:35 beta-nexus.******.tld dbus-daemon[599]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.365' (uid=0 pid=65537 comm="/usr/bin/hostnamectl " label="unconfined")
Aug 03 06:36:35 beta-nexus.******.tld systemd[1]: Starting Hostname Service...
-- Subject: A start job for unit systemd-hostnamed.service has begun execution
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- A start job for unit systemd-hostnamed.service has begun execution.
-- 
-- The job identifier is 7793.
Aug 03 06:36:35 beta-nexus.******.tld dbus-daemon[599]: [system] Successfully activated service 'org.freedesktop.hostname1'
Aug 03 06:36:35 beta-nexus.******.tld systemd[1]: Started Hostname Service.
-- Subject: A start job for unit systemd-hostnamed.service has finished successfully
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- A start job for unit systemd-hostnamed.service has finished successfully.
-- 
-- The job identifier is 7793.
Aug 03 06:36:39 beta-nexus.******.tld dbus-daemon[599]: [system] Activating via systemd: service name='org.freedesktop.PackageKit' unit='packagekit.service' requested by ':1.367' (uid=0 pid=66327 comm="/usr/bin/gdbus call --system --dest org.freedeskto" label="unconfined")
Aug 03 06:36:39 beta-nexus.******.tld systemd[1]: Starting PackageKit Daemon...
-- Subject: A start job for unit packagekit.service has begun execution
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- A start job for unit packagekit.service has begun execution.
-- 
-- The job identifier is 7861.
Aug 03 06:36:39 beta-nexus.******.tld PackageKit[66330]: daemon start
Aug 03 06:36:39 beta-nexus.******.tld dbus-daemon[599]: [system] Successfully activated service 'org.freedesktop.PackageKit'
Aug 03 06:36:39 beta-nexus.******.tld systemd[1]: Started PackageKit Daemon.
-- Subject: A start job for unit packagekit.service has finished successfully
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- A start job for unit packagekit.service has finished successfully.
-- 
-- The job identifier is 7861.
Aug 03 06:36:46 beta-nexus.******.tld systemd[1]: Stopping Nexus Service...
-- Subject: A stop job for unit nexus.service has begun execution
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- A stop job for unit nexus.service has begun execution.
-- 
-- The job identifier is 7929.
Aug 03 06:36:46 beta-nexus.******.tld nexus[66504]: Shutting down nexus
Aug 03 06:36:49 beta-nexus.******.tld nexus[66504]: Stopped.
Aug 03 06:36:49 beta-nexus.******.tld systemd[1]: nexus.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- The unit nexus.service has successfully entered the 'dead' state.
Aug 03 06:36:49 beta-nexus.******.tld systemd[1]: Stopped Nexus Service.
-- Subject: A stop job for unit nexus.service has finished
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- A stop job for unit nexus.service has finished.
-- 
-- The job identifier is 7929 and the job result is done.
Aug 03 06:36:51 beta-nexus.******.tld systemd[1]: Starting Nexus Service...
-- Subject: A start job for unit nexus.service has begun execution
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- A start job for unit nexus.service has begun execution.
-- 
-- The job identifier is 7930.
Aug 03 06:36:51 beta-nexus.******.tld nexus[66763]: Starting nexus
Aug 03 06:36:51 beta-nexus.******.tld systemd[1]: Started Nexus Service.
-- Subject: A start job for unit nexus.service has finished successfully
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- A start job for unit nexus.service has finished successfully.
-- 
-- The job identifier is 7930.
Aug 03 06:37:05 beta-nexus.******.tld systemd[1]: systemd-hostnamed.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- The unit systemd-hostnamed.service has successfully entered the 'dead' state.
Aug 03 06:41:44 beta-nexus.******.tld PackageKit[66330]: daemon quit
Aug 03 06:41:44 beta-nexus.******.tld systemd[1]: packagekit.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- The unit packagekit.service has successfully entered the 'dead' state.

答案1

好像和packagekit有关。 Nexus的重启是在packagekit启动后5秒左右。

我想知道你是否能弄清楚这个PID是什么:

Aug 2 11:54:48 beta-nexus dbus-daemon[585]: ... (uid=0 pid=367843 comm="/usr/bin/gdbus call --system --dest org.freedeskto" label="unconfined")

相关内容