完全重新安装 kubernetes 和 kubectl 插件后 kube-apiserver.service 失败

完全重新安装 kubernetes 和 kubectl 插件后 kube-apiserver.service 失败

我遇到一个问题,kube-apiserver.service 在我的本地 Fedora 36 上总是失败。

从上下文中获取命名空间时,我遇到了证书问题,导致我无法成功。我使用时kubens出现错误:

> error: You must be logged in to the server (Unauthorized) 
> error getting namespace list

我首先检查了我的 ~/.kube/config,一切似乎都正常。因此,在阅读了一些内容后,我确信这是一个证书错误(我们在使用特定 kube 集群时遇到了证书错误),我kubeadm通过 yum ( sudo yum install kubernetes-kubeadm.x86_64) 进行了安装。我使用它通过命令自动续订了所有需要的证书kubeadm certs renew all

该命令的输出很干净,没有错误信号。检查 kubens 仍然会出现相同的错误。因此,我尝试重新启动 kube 服务,除了 kube-apiserver 之外,所有服务都重新启动正常。它总是出现相同的错误,太多的重启命令重复得太快。这是 的输出sudo systemctl status kube-apiserver -l

> × kube-apiserver.service - Kubernetes API Server
>      Loaded: loaded (/usr/lib/systemd/system/kube-apiserver.service; enabled; vendor preset: disabled)
>      Active: failed (Result: exit-code) since Thu 2022-11-17 09:07:44 CET; 12min ago
>        Docs: https://kubernetes.io/docs/concepts/overview/components/#kube-apiserver
>              https://kubernetes.io/docs/reference/generated/kube-apiserver/
>     Process: 1752 ExecStart=/usr/bin/kube-apiserver $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBE_ETCD_SERVERS $KUBE_API_ADDRESS $KUBE_API_PORT
> $KUBELET_PORT >    Main PID: 1752 (code=exited, status=1/FAILURE)
>         CPU: 48ms
> 
> Nov 17 09:07:44 fedora systemd[1]: kube-apiserver.service: Scheduled
> restart job, restart counter is at 5. Nov 17 09:07:44 fedora
> systemd[1]: Stopped kube-apiserver.service - Kubernetes API Server.
> Nov 17 09:07:44 fedora systemd[1]: kube-apiserver.service: Start
> request repeated too quickly. Nov 17 09:07:44 fedora systemd[1]:
> kube-apiserver.service: Failed with result 'exit-code'. Nov 17
> 09:07:44 fedora systemd[1]: Failed to start kube-apiserver.service -
> Kubernetes API Server.

因此我查看了 journalctl 并发现了这个日志部分:

>     Nov 16 16:33:30 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0
> msg='unit=kube-apiserver comm="systemd" exe="/usr/lib/systemd/systemd"
> hostname=? addr=? terminal=? res=failed'
>     Nov 16 16:33:30 fedora systemd[1]: kube-apiserver.service: Scheduled restart job, restart counter is at 5.
>     ░░ Automatic restarting of the unit kube-apiserver.service has been scheduled, as the result for
>     Nov 16 16:33:30 fedora systemd[1]: Stopped kube-apiserver.service - Kubernetes API Server.
>     ░░ Subject: A stop job for unit kube-apiserver.service has finished
>     ░░ A stop job for unit kube-apiserver.service has finished.
>     Nov 16 16:33:30 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0
> msg='unit=kube-apiserver comm="systemd" exe="/usr/lib/systemd/systemd"
> hostname=? addr=? terminal=? res=success'
>     Nov 16 16:33:30 fedora audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0
> msg='unit=kube-apiserver comm="systemd" exe="/usr/lib/systemd/systemd"
> hostname=? addr=? terminal=? res=success'
>     Nov 16 16:33:30 fedora systemd[1]: kube-apiserver.service: Start request repeated too quickly.
>     Nov 16 16:33:30 fedora systemd[1]: kube-apiserver.service: Failed with result 'exit-code'.
>     ░░ The unit kube-apiserver.service has entered the 'failed' state with result 'exit-code'.
>     Nov 16 16:33:30 fedora systemd[1]: Failed to start kube-apiserver.service - Kubernetes API Server.
>     ░░ Subject: A start job for unit kube-apiserver.service has failed
>     ░░ A start job for unit kube-apiserver.service has finished with a failure.
>     Nov 16 16:33:37 fedora kubelet[8800]:       --rotate-certificates                                      <Warning: Beta feature> Auto rotate the kubelet client certificates by
> requesting new certificates from the kube-apiserver when the
> certificate expiration approaches. (DEPRECATED: This parameter should
> be set via the config file specified by the Kubelet's --config flag.
> See
> https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/
> for more information.)
>     Nov 16 16:33:37 fedora kubelet[8800]:       --rotate-server-certificates                               Auto-request and rotate the kubelet serving certificates by requesting
> new certificates from the kube-apiserver when the certificate
> expiration approaches. Requires the RotateKubeletServerCertificate
> feature gate to be enabled, and approval of the submitted
> CertificateSigningRequest objects. (DEPRECATED: This parameter should
> be set via the config file specified by the Kubelet's --config flag.
> See
> https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/
> for more information.)
>     Nov 16 16:33:47 fedora kubelet[8818]:       --rotate-certificates                                      <Warning: Beta feature> Auto rotate the kubelet client certificates by
> requesting new certificates from the kube-apiserver when the
> certificate expiration approaches. (DEPRECATED: This parameter should
> be set via the config file specified by the Kubelet's --config flag.
> See
> https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/
> for more information.)
>     Nov 16 16:33:47 fedora kubelet[8818]:       --rotate-server-certificates                               Auto-request and rotate the kubelet serving certificates by requesting
> new certificates from the kube-apiserver when the certificate
> expiration approaches. Requires the RotateKubeletServerCertificate
> feature gate to be enabled, and approval of the submitted
> CertificateSigningRequest objects. (DEPRECATED: This parameter should
> be set via the config file specified by the Kubelet's --config flag.
> See
> https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/
> for more information.)
>     Nov 16 16:33:57 fedora kubelet[8834]:       --rotate-certificates                                      <Warning: Beta feature> Auto rotate the kubelet client certificates by
> requesting new certificates from the kube-apiserver when the
> certificate expiration approaches. (DEPRECATED: This parameter should
> be set via the config file specified by the Kubelet's --config flag.
> See
> https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/
> for more information.)
>     Nov 16 16:33:57 fedora kubelet[8834]:       --rotate-server-certificates                               Auto-request and rotate the kubelet serving certificates by requesting
> new certificates from the kube-apiserver when the certificate
> expiration approaches. Requires the RotateKubeletServerCertificate
> feature gate to be enabled, and approval of the submitted
> CertificateSigningRequest objects. (DEPRECATED: This parameter should
> be set via the config file specified by the Kubelet's --config flag.
> See
> https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/
> for more information.)

的输出kubectl version为:

>     Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.7",
> GitCommit:"e6f35974b08862a23e7f4aad8e5d7f7f2de26c15",
> GitTreeState:"archive", BuildDate:"2022-10-14T00:00:00Z",
> GoVersion:"go1.18.7", Compiler:"gc", Platform:"linux/amd64"}
>     Kustomize Version: v4.5.4
>     error: You must be logged in to the server (the server has asked for the client to provide credentials)

(是的,其中有一个错误消息)。

真的不知道接下来该怎么办。您会尝试什么来让 kube-apiserver.service 恢复正常?

我尝试卸载系统上能找到的每个 kubernetes 包:

sudo rpm -e kubernetes-client-1.24.7-1.fc36.x86_64 kubernetes-1.24.7-1.fc36.x86_64 kubernetes-master-1.24.7-1.fc36.x86_64
kubernetes-node-1.24.7-1.fc36.x86_64 

通过 krew 删除所有 kubectl 插件后。然后我备份了我的 .kube/config 并将名称更改为整个 ~/.kube 文件夹。我重新安装了 kubernetes,此时kubectl version返回端口 8080 错误,我认为这一定是因为我还没有 .kube/config。我重新安装了 krew 和我最喜欢的 kubectl 插件(ctx、ns、cm),并为我需要访问的所有 kubernetes 集群重建了配置(使用aws eks update-kubeconfigkubecm add -f <file>命令)。现在 kubectl 版本具有更正常的输出:

> Client Version: version.Info{Major:"1", Minor:"24",
> GitVersion:"v1.24.7",
> GitCommit:"e6f35974b08862a23e7f4aad8e5d7f7f2de26c15",
> GitTreeState:"archive", BuildDate:"2022-10-14T00:00:00Z",
> GoVersion:"go1.18.7", Compiler:"gc", Platform:"linux/amd64"} Kustomize
> Version: v4.5.4 Server Version: version.Info{Major:"1", Minor:"21+",
> GitVersion:"v1.21.14-eks-fb459a0",
> GitCommit:"b07006b2e59857b13fe5057a956e86225f0e82b7",
> GitTreeState:"clean", BuildDate:"2022-10-24T20:32:54Z",
> GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"} WARNING:
> version difference between client (1.24) and server (1.21) exceeds the
> supported minor version skew of +/-1

运行只会 sudo kube-apiserver给出输出:

> W1117 10:13:55.819927   16008 services.go:37] No CIDR for service
> cluster IPs specified. Default value which was 10.0.0.0/24 is
> deprecated and will be removed in future releases. Please specify it
> using --service-cluster-ip-range on kube-apiserver. I1117
> 10:13:56.031051   16008 serving.go:342] Generated self-signed cert
> (/var/run/kubernetes/apiserver.crt, /var/run/kubernetes/apiserver.key)
> I1117 10:13:56.031063   16008 server.go:558] external host was not
> specified, using 192.168.XX.XX W1117 10:13:56.031069   16008
> authentication.go:526] AnonymousAuth is not allowed with the
> AlwaysAllow authorizer. Resetting AnonymousAuth to false. You should
> use a different authorizer E1117 10:13:56.031184   16008 run.go:74]
> "command failed" err="[--etcd-servers must be specified,
> service-account-issuer is a required flag,
> --service-account-signing-key-file and --service-account-issuer are required flags]"

sudo systemctl status kube-apiserver 仍然显示失败状态,并且 sudo systemctl restart kube-apiserver 仍然导致失败

相关内容