☰

Kubectl 与私有 EKS 集群生成 TLS 握手超时

tag-icon • tag-icon ssl amazon-eks kubectl
Kubectl 与私有 EKS 集群生成 TLS 握手超时

运行任何 kubectl 命令时，我看到以下错误，并且没有返回任何数据。通过 VPN 连接访问私有 AWS EKS 实例时会发生此错误。

$ kubectl get pods -A  -v=9 
...
5800 helpers.go:116] Unable to connect to the server: net/http: TLS handshake timeout

关于错误的奇怪之处在于 kubectl 会产生错误，但是输出所有 pod 数据第一次运行 kubectl如果没有发现缓存。但之后 kubectl 会失败并且不返回任何数据。如果我删除缓存目录（rm -rf ~/.kube/cache），kubectl 会工作一次，然后由于重新创建了 ~/.kube/cache 而再次失败。

例如，我第一次运行 kubectl：

$ kubectl get pods -A  -v=9 
I0718 14:52:58.797861   15292 loader.go:372] Config loaded from file:  U:\.kube\config
I0718 14:52:58.806839   15292 round_trippers.go:435] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl.exe/v1.22.0 (wi
ndows/amd64) kubernetes/c2b5237" 'https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=32s'
I0718 14:53:13.037830   15292 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=
32s  in 14230 milliseconds
I0718 14:53:13.038981   15292 round_trippers.go:460] Response Headers:
I0718 14:53:13.044027   15292 cached_discovery.go:121] skipped caching discovery info due to Get "https://C21D1C150B2FC9F1252A79875E11C4BC.gr7
.us-east-2.eks.amazonaws.com/api?timeout=32s": net/http: TLS handshake timeout
I0718 14:53:13.051169   15292 round_trippers.go:435] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl.exe/v1.22.0 (wi
ndows/amd64) kubernetes/c2b5237" 'https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=32s'
I0718 14:53:23.063199   15292 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=
32s  in 10010 milliseconds
I0718 14:53:23.065975   15292 round_trippers.go:460] Response Headers:
I0718 14:53:23.065975   15292 cached_discovery.go:121] skipped caching discovery info due to Get "https://C21D1C150B2FC9F1252A79875E11C4BC.gr7
.us-east-2.eks.amazonaws.com/api?timeout=32s": net/http: TLS handshake timeout
I0718 14:53:23.114872   15292 shortcut.go:89] Error loading discovery information: Get "https://C21D1C150B2FC9F1252A79875E11C4BC.gr7.us-east-2
.eks.amazonaws.com/api?timeout=32s": net/http: TLS handshake timeout
I0718 14:53:23.114872   15292 round_trippers.go:435] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl.exe/v1.22.0 (wi
ndows/amd64) kubernetes/c2b5237" 'https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=32s'
I0718 14:53:23.266940   15292 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=
32s 200 OK in 152 milliseconds
I0718 14:53:23.267518   15292 round_trippers.go:460] Response Headers:
I0718 14:53:23.268082   15292 round_trippers.go:463]     Content-Type: application/json
I0718 14:53:23.268082   15292 round_trippers.go:463]     Content-Length: 166
I0718 14:53:23.268082   15292 round_trippers.go:463]     Date: Mon, 18 Jul 2022 19:53:23 GMT
I0718 14:53:23.268649   15292 round_trippers.go:463]     Audit-Id: dfc5cfe6-08d5-46a8-a61c-632dc3a21613
I0718 14:53:23.268649   15292 round_trippers.go:463]     Cache-Control: no-cache, private
I0718 14:53:23.307493   15292 request.go:1181] Response Body: {"kind":"APIVersions","versions":["v1"],"serverAddressByClientCIDRs":[{"clientCI
DR":"0.0.0.0/0","serverAddress":"ip-10-10-1-1.us-east-2.compute.internal:443"}]}
I0718 14:53:23.336044   15292 round_trippers.go:435] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl.exe/v1.22.0 (wi
ndows/amd64) kubernetes/c2b5237" 'https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/apis?timeout=32s'
I0718 14:53:23.368489   15292 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/apis?timeout
=32s 200 OK in 32 milliseconds
I0718 14:53:23.369867   15292 round_trippers.go:460] Response Headers:
I0718 14:53:23.369867   15292 round_trippers.go:463]     Cache-Control: no-cache, private
I0718 14:53:23.369867   15292 round_trippers.go:463]     Content-Type: application/json
I0718 14:53:23.369867   15292 round_trippers.go:463]     Date: Mon, 18 Jul 2022 19:53:23 GMT
I0718 14:53:23.369867   15292 round_trippers.go:463]     Audit-Id: ba3c50bf-66a3-411e-8763-ec302cc78d03
...

该命令返回 pod 数据。我注意到，在 http 返回 200 OK 之前，需要进行 3 次 curl 尝试，从那时起，所有 curl 命令似乎都成功了。

此后，如果我运行另一个 kubectl 命令，我会得到以下错误输出并且没有 pod 数据：

$ kubectl get pods -A  -v=9 --insecure-skip-tls-verify=true
I0718 14:51:33.249188    1640 loader.go:372] Config loaded from file:  U:\.kube\config
I0718 14:51:33.427333    1640 round_trippers.go:435] curl -v -XGET  -H "Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json;
as=Table;v=v1beta1;g=meta.k8s.io,application/json" -H "User-Agent: kubectl.exe/v1.22.0 (windows/amd64) kubernetes/c2b5237" 'https://C21D1C150B
2FC9F1252A79875E11C4BC.gr7.us-east-2.eks.amazonaws.com/api/v1/pods?limit=500'
I0718 14:51:47.439207    1640 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api/v1/pods?
limit=500  in 14011 milliseconds
I0718 14:51:47.440457    1640 round_trippers.go:460] Response Headers:
I0718 14:51:47.453797    1640 helpers.go:235] Connection error: Get https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/a
pi/v1/pods?limit=500: net/http: TLS handshake timeout
F0718 14:51:47.453797    1640 helpers.go:116] Unable to connect to the server: net/http: TLS handshake timeout
goroutine 1 [running]:
k8s.io/kubernetes/vendor/k8s.io/klog/v2.stacks(0xc0000d4001, 0xc000804000, 0x6f, 0xf9)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1026 +0xbf
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).output(0x300ff60, 0xc000000003, 0x0, 0x0, 0xc00012c0e0, 0x2, 0x271bb69, 0xa, 0x74, 0x2bef0
0)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:975 +0x1fb
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).printDepth(0x300ff60, 0xc000000003, 0x0, 0x0, 0x0, 0x0, 0x2, 0xc000788270, 0x1, 0x1)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:735 +0x190
k8s.io/kubernetes/vendor/k8s.io/klog/v2.FatalDepth(...)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1500
k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util.fatal(0xc0000af450, 0x41, 0x1)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util/helpers.go:94 +0x296
k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util.checkErr(0x218bc20, 0xc000004198, 0x2003930)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util/helpers.go:178 +0x8b5
k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util.CheckErr(...)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util/helpers.go:116
k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/get.NewCmdGet.func2(0xc000376280, 0xc0000dc880, 0x1, 0x4)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/get/get.go:180 +0x15d
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute(0xc000376280, 0xc0000dc840, 0x4, 0x4, 0xc000376280, 0xc0000dc840)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:856 +0x2c2
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc0003bcc80, 0xc0000e0000, 0xc0000de000, 0x6)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:960 +0x375
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute(...)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:897
main.main()
        _output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/kubectl.go:49 +0x234

goroutine 19 [chan receive]:
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).flushDaemon(0x300ff60)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1169 +0x92
created by k8s.io/kubernetes/vendor/k8s.io/klog/v2.init.0
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:420 +0xe5

goroutine 21 [select]:
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x2003838, 0x2189500, 0xc000574000, 0x6c612079786f7201, 0xc000082ba0)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:167 +0x1
19
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x2003838, 0x12a05f200, 0x0, 0x6c74636562756b01, 0xc000082ba0)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x9
f
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.Until(0x2003838, 0x12a05f200, 0xc000082ba0)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x54
created by k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/util/logs.InitLogs
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/util/logs/logs.go:51 +0x9e

我尝试设置 NO_PROXY 环境变量，但没有帮助。

有什么想法吗？

相关内容