无法在 Proxmox VM 上安装 K3s

无法在 Proxmox VM 上安装 K3s

我正在尝试使用 Proxmox 和一小组 Raspberriy PIs 4B 创建一个 HA k3s 集群。

对于 PI 来说,一切正常,但是当尝试在 Proxmox VM 上安装主机时,它将无法启动。

我的设置:

  • 主机:Proxmox 7.0.7(我也尝试过 6.4.4)
  • 访客:Ubuntu 20.04.2
  • K3S:v1.21.3+k3s1(我也尝试过 v1.19.13+k3s1)
  • MariaDB:10.3

我正在运行这些命令来安装主

export K3S_DATASTORE_ENDPOINT='mysql://DB_USER:DB_PASSWORD@tcp(DB_IP:DB_PORT)/DB_SCHEME'

curl -sfL https://get.k3s.io | sh -s - server --node-taint CriticalAddonsOnly=true:NoExecute --tls-san NGINX_LOADBALANCER_IP

这是安装和启动的输出:

[INFO]  Using v1.19.13+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.19.1                                                                                                                                                             3+k3s1/sha256sum-amd64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.19                                                                                                                                                             .13+k3s1/k3s
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Creating /usr/local/bin/ctr symlink to k3s
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s
Job for k3s.service failed because the control process exited with error code.
See "systemctl status k3s.service" and "journalctl -xe" for details.

我检查了日志:

systemctl status k3s.service


● k3s.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Tue 2021-08-03 20:27:40 UTC; 2s ago
       Docs: https://k3s.io
    Process: 6181 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
    Process: 6193 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
    Process: 6194 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
    Process: 6195 ExecStart=/usr/local/bin/k3s server --node-taint CriticalAddonsOnly=true:NoExecute --tls-san NGINX_LOADBALANNCER_IP (code=exited, status=1/FAILURE)
   Main PID: 6195 (code=exited, status=1/FAILURE)

Aug 03 20:27:40 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:27:40 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:27:40 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.

和:

journalctl -u k3s.service


Aug 03 20:06:17 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:17 k3svm1 sh[19450]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:17 k3svm1 sh[19451]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 03 20:06:17 k3svm1 k3s[19460]: time="2021-08-03T20:06:17Z" level=info msg="Acquiring lock file /var/lib/rancher/k3s/data/.lock"
Aug 03 20:06:17 k3svm1 k3s[19460]: time="2021-08-03T20:06:17Z" level=info msg="Preparing data dir /var/lib/rancher/k3s/data/9df574741d2573cbbe6616e8624488b36b3340d077bc50da7cb167f1b08a64d1"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.748397535Z" level=info msg="Starting k3s v1.21.3+k3s1 (1d1f220f)"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.751745749Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.751876220Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.753348552Z" level=info msg="Database tables and indexes are up to date"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.757714719Z" level=info msg="Kine listening on unix://kine.sock"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.764631916Z" level=info msg="certificate CN=system:admin,O=system:masters signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.765377675Z" level=info msg="certificate CN=system:kube-controller-manager signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.766187231Z" level=info msg="certificate CN=system:kube-scheduler signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.766815165Z" level=info msg="certificate CN=kube-apiserver signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.767415198Z" level=info msg="certificate CN=system:kube-proxy signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 >
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.767950031Z" level=info msg="certificate CN=system:k3s-controller signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.768698847Z" level=info msg="certificate CN=k3s-cloud-controller-manager signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:0>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.769745716Z" level=info msg="certificate CN=kube-apiserver signed by CN=k3s-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.770870630Z" level=info msg="certificate CN=system:auth-proxy signed by CN=k3s-request-header-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:1>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.771882180Z" level=info msg="certificate CN=etcd-server signed by CN=etcd-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.772508382Z" level=info msg="certificate CN=etcd-client signed by CN=etcd-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.773399505Z" level=info msg="certificate CN=etcd-peer signed by CN=etcd-peer-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.813171353Z" level=info msg="certificate CN=k3s,O=k3s signed by CN=k3s-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.813556476Z" level=info msg="Active TLS secret  (ver=) (count 9): map[listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn->
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.819862032Z" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Aug 03 20:06:18 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:06:18 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:06:18 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
Aug 03 20:06:23 k3svm1 systemd[1]: k3s.service: Scheduled restart job, restart counter is at 1.
Aug 03 20:06:23 k3svm1 systemd[1]: Stopped Lightweight Kubernetes.
Aug 03 20:06:23 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:23 k3svm1 sh[19478]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:23 k3svm1 sh[19483]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.115279840Z" level=info msg="Starting k3s v1.21.3+k3s1 (1d1f220f)"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.119390931Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.119554649Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.121305745Z" level=info msg="Database tables and indexes are up to date"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.125898745Z" level=info msg="Kine listening on unix://kine.sock"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.146164308Z" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Aug 03 20:06:24 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:06:24 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:06:24 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
Aug 03 20:06:29 k3svm1 systemd[1]: k3s.service: Scheduled restart job, restart counter is at 2.
Aug 03 20:06:29 k3svm1 systemd[1]: Stopped Lightweight Kubernetes.
Aug 03 20:06:29 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:29 k3svm1 sh[19507]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:29 k3svm1 sh[19508]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.565328025Z" level=info msg="Starting k3s v1.21.3+k3s1 (1d1f220f)"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.568959518Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.568994906Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.570693830Z" level=info msg="Database tables and indexes are up to date"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.575194321Z" level=info msg="Kine listening on unix://kine.sock"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.594809727Z" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Aug 03 20:06:29 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:06:29 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:06:29 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
Aug 03 20:06:34 k3svm1 systemd[1]: k3s.service: Scheduled restart job, restart counter is at 3.
Aug 03 20:06:34 k3svm1 systemd[1]: Stopped Lightweight Kubernetes.
Aug 03 20:06:34 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:34 k3svm1 sh[19527]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:34 k3svm1 sh[19530]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory

从那里继续重启。

如上所述,我尝试了多个版本,但似乎都没有用。而且我并没有真正收到错误。我在不同的 GitHub 问题中找到的唯一提示是通过编辑 /boot/cmdline.txt 启用有关 Raspberry PI 的容器化。但是,我没有在 PI 上遇到问题,而是在 Proxmox-VM 上遇到问题。

我是否遗漏了什么?

不知何故这家伙设法让它在相同的设置下运行。有其他人让它运行过吗?可以提供一些参考吗?

相关内容