我目前正在运行一个 k8s 集群设置,其中安装了 3 个主节点和 3 个工作节点,并安装了 Kubeadm。每个主节点负责管理一个工作节点。在我的一个工作节点中,最近出现了一个以前从未见过的错误。
I0226 10:03:50.904111 1066929 round_trippers.go:510] HTTP Trace: Dial to tcp:127.0.0.1:6443 succeed
I0226 10:03:50.918311 1066929 round_trippers.go:553] GET https://127.0.0.1:6443/api/v1/namespaces/kube-system/pods/kube-proxy-ssss2 200 OK in 15 milliseconds
I0226 10:03:50.918395 1066929 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 8 ms ServerProcessing 4 ms Duration 15 ms
I0226 10:03:50.918447 1066929 round_trippers.go:577] Response Headers:
I0226 10:03:50.918478 1066929 round_trippers.go:580] Date: Mon, 26 Feb 2024 10:03:50 GMT
I0226 10:03:50.918529 1066929 round_trippers.go:580] Audit-Id: a7c16d6f-3cb0-48a8-aa73-e2e4fbffde64
I0226 10:03:50.918559 1066929 round_trippers.go:580] Cache-Control: no-cache, private
I0226 10:03:50.918579 1066929 round_trippers.go:580] Content-Type: application/json
I0226 10:03:50.918639 1066929 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 1cafbc8f-a46e-4e74-9804-03b72b0beb1c
I0226 10:03:50.918661 1066929 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: 447a939f-c32c-455a-bc49-1bc69b1544e4
I0226 10:03:50.918906 1066929 request.go:1171] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"kube-proxy-ssss2","generateName":"kube-proxy-","namespace":"kube-system","uid":"be28ae46-b701-4466-ade5-01032008209d","resourceVersion":"117095929","creationTimestamp":"2024-02-26T04:16:43Z","labels":{"controller-revision-hash":"6bc4695d8c","k8s-app":"kube-proxy","pod-template-generation":"1"},"ownerReferences":[{"apiVersion":"apps/v1","kind":"DaemonSet","name":"kube-proxy","uid":"15ecadb4-5d6c-4fb4-8f51-36dbb9c1a5a4","controller":true,"blockOwnerDeletion":true}],"managedFields":[{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2024-02-26T04:16:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},
I0226 10:03:50.947158 1066929 round_trippers.go:580] Audit-Id: fda7a277-2512-4d58-a1a8-7bc48e4b1752
I0226 10:03:50.947183 1066929 round_trippers.go:580] Cache-Control: no-cache, private
I0226 10:03:50.947202 1066929 round_trippers.go:580] Content-Type: text/plain
I0226 10:03:50.947222 1066929 round_trippers.go:580] Date: Mon, 26 Feb 2024 10:03:50 GMT
E0226 04:16:43.854878 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused
E0226 04:16:45.010666 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused
E0226 04:16:47.300738 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused
E0226 04:16:51.658415 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused
E0226 04:17:01.247069 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused
E0226 04:17:19.267198 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused
I0226 04:17:19.267293 1 server.go:820] "Can't determine this node's IP, assuming 127.0.0.1; if this is incorrect, please set the --bind-address flag"
I0226 04:17:19.267352 1 server_others.go:109] "Detected node IP" address="127.0.0.1"
I0226 04:17:19.304643 1 server_others.go:248] "Using ipvs Proxier"
主节点中的 Kube Proxy pod 工作正常,但工作节点中的 pod 则不行。我尝试重置工作节点并将其重新加入主节点。似乎没有任何效果,Kube-Proxy pod 仍然无法确定其节点 IP 地址应该是什么。
检索似乎https://127.0.0.1:6443/api/v1/namespaces/kube-system/pods/kube-proxy-ssss2
运行良好,但调用https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03
产生:connect connection refused
:。
我认为这可能是 pod 找不到其节点的原因,但我不太清楚为什么会发生这种情况。任何帮助或见解都将不胜感激
答案1
不过,我们能够通过更改 kube-proxy 中的 config-map 使其直接指向 api 服务器来实现这一点,而且它似乎已经起作用了。遇到此问题的人请确保您的 kube-proxy 指向正确的服务器。