我遇到了一个相当奇怪的问题,我不知道该如何理解它。看起来,在 mysql systemd 服务已经运行后,mysqld 进程正在由 systemd 以外的其他程序使用不同的参数启动。这导致 mysql 服务器关闭,新进程变得无响应。
通过 systemd 启动时:
/usr/sbin/mysqld
什么时候开始的?
mysqld --user=mysql --init-file=/var/lib/mysql-files/tmp.Ld6xyAe3qb --socket=/tmp/tmp.obBArlPCj2/mysqld.sock --pid-file=/tmp/tmp.obBArlPCj2/mysqld.pid
目前还不清楚为什么要重新启动 mysql、由谁重新启动以及新的参数来自哪里。
/var/lib/mysql-files/tmp.Ld6xyAe3qb
(即上面提供的 init 文件)包含以下文本:
USE mysql;
ALTER USER 'root'@'localhost' IDENTIFIED WITH 'auth_socket';
SHUTDOWN;
我在日志中唯一能看到的是:
2020-08-08T06:57:31.123013Z 0 [System] [MY-013172] [Server] Received SHUTDOWN from user <via user signal>. Shutting down mysqld (Version: 8.0.21-0ubuntu0.20.04.4).
... [group replication, InnoDB, normal logs] ...
2020-08-08T06:57:49.954754Z 8 [System] [MY-013172] [Server] Received SHUTDOWN from user boot. Shutting down mysqld (Version: 8.0.21-0ubuntu0.20.04.4).
第一个日志是随机出现的;它之前没有任何内容表明存在内存或任何其他类型的错误(我能看到)。第二个关闭似乎来自 init 文件。然而,更令人困惑的是boot
系统上或 mysql 中没有用户。
唯一使用 mysql 的服务是 OpenStack Keystone,用于 Swift 对象存储,没有其他 OpenStack 服务。
据我所知,没有其他 cron 或 systemd 服务引用 mysql,但我对该测试不确定。命令是:
systemctl list-units --all --no-legend | awk '{print $1}' | xargs -n1 systemctl cat 2>/dev/null | grep mysql
它给出:
After=postgresql.service mysql.service keystone.service rabbitmq-server.service ntp.service network-online.target local-fs.target remote-fs.target
# /lib/systemd/system/mysql.service
User=mysql
Group=mysql
PIDFile=/run/mysqld/mysqld.pid
ExecStartPre=/usr/share/mysql/mysql-systemd-start pre
ExecStart=/usr/sbin/mysqld
RuntimeDirectory=mysqld
第一行来自/lib/systemd/system/cinder-volume.service
,需要删除,但我不认为与此问题相关,因为问题发生在从未安装过 cinder 的服务器上
该问题出现在设置了组复制的 4 台服务器上。
# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04 LTS
Release: 20.04
Codename: focal
# mysql --version
mysql Ver 8.0.21-0ubuntu0.20.04.4 for Linux on x86_64 ((Ubuntu))
# cat /etc/mysql/mysql.conf.d/mysqld.cnf
#
# The MySQL database server configuration file.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html
# Here is entries for some specific programs
# The following values assume you have at least 32M ram
[mysqld]
log-error-verbosity = 5
#
# * Basic Settings
#
user = mysql
# pid-file = /var/run/mysqld/mysqld.pid
# socket = /var/run/mysqld/mysqld.sock
# port = 3306
# datadir = /var/lib/mysql
# If MySQL is running as a replication slave, this should be
# changed. Ref https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_tmpdir
# tmpdir = /tmp
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 0.0.0.0
#
# * Fine Tuning
#
key_buffer_size = 16M
# max_allowed_packet = 64M
# thread_stack = 256K
# thread_cache_size = -1
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover-options = BACKUP
# max_connections = 151
# table_open_cache = 4000
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
#
# Log all queries
# Be aware that this log type is a performance killer.
# general_log_file = /var/log/mysql/query.log
# general_log = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
# slow_query_log = 1
# slow_query_log_file = /var/log/mysql/mysql-slow.log
# long_query_time = 2
# log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
# server-id = 1
# log_bin = /var/log/mysql/mysql-bin.log
# binlog_expire_logs_seconds = 2592000
max_binlog_size = 100M
# binlog_do_db = include_database_name
# binlog_ignore_db = include_database_name
# disable non innodb storage engines for group replication
# https://dev.mysql.com/doc/refman/8.0/en/group-replication-configuring-instances.html
disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY"
# use native password auth
default_authentication_plugin=mysql_native_password
# replication framework settings
server_id = 1
gtid_mode = ON
enforce_gtid_consistency = ON
log_bin = binlog
log_slave_updates = ON
binlog_format = ROW
master_info_repository = TABLE
relay_log_info_repository = TABLE
binlog_checksum = NONE
# group replication settings
plugin_load_add = 'group_replication.so'
plugin-load-add = 'mysql_clone.so'
group_replication_group_name = "xxx"
group_replication_start_on_boot = on
group_replication_local_address = "xxx:33061"
group_replication_ip_whitelist="xxx"
group_replication_group_seeds = "xxx:33061"
group_replication_bootstrap_group = off
group_replication_enforce_update_everywhere_checks = off
group_replication_single_primary_mode = off
## tls
# required encrypted remote connections
# require_secure_transport = ON
tls_version=TLSv1.3
ssl_ca = "/etc/mysql/certs/eons.dev.pem"
ssl_cert = "/etc/mysql/certs/skyrim-mysql.pem"
ssl_key = "/etc/mysql/certs/skyrim-mysql.key"
ssl_cipher = "DHE-RSA-AES256-SHA"
##ssl_crl = "crl-server-revoked.crl"
##ssl_crlpath = "/.../crl_directory"
mysqlx_ssl_ca = "/etc/mysql/certs/eons.dev.pem"
mysqlx_ssl_cert = "/etc/mysql/certs/skyrim-mysql.pem"
mysqlx_ssl_key = "/etc/mysql/certs/skyrim-mysql.key"
mysqlx_ssl_cipher = "DHE-RSA-AES256-SHA"
# group replication tls
group_replication_ssl_mode = REQUIRED
group_replication_recovery_use_ssl = on
group_replication_recovery_ssl_ca = "/etc/mysql/certs/eons.dev.pem"
group_replication_recovery_ssl_cert = "/etc/mysql/certs/skyrim-mysql.pem"
group_replication_recovery_ssl_key = "/etc/mysql/certs/skyrim-mysql.key"
group_replication_recovery_ssl_cipher = "DHE-RSA-AES256-SHA"