我在全新的 Ubuntu Server 20.04 上使用 Rancher 部署了一个新的 kubernetes 集群(版本 1.18)。我面临的问题是,从任何 pod 发出的每个 https 请求都会失败,并显示以下消息:
curl -v https://pypi.org/simple/pip/
* Trying 44.227.65.245...
* Connected to pypi.org (44.227.65.245) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 692 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* gnutls_handshake() failed: Handshake failed
* Closing connection 0
curl: (35) gnutls_handshake() failed: Handshake failed
我已经测试过了
kubectl create deployment hello-node --image=k8s.gcr.io/echoserver:1.4
kubectl exec -it <pod_id> bash
我在主机系统上通过 ssh 尝试了同样的事情,一切正常。我还尝试使用与 pod 相同的映像运行独立的 docker 容器(在同一台机器上),docker run -it --rm k8s.gcr.io/echoserver:1.4 bash
这里也没有问题。所以我猜这与我的 kubernetes 设置有关。
它只发生在 https 请求中 - http 请求工作正常。
我尝试删除该集群并从头开始准备它,但这没有帮助。
更新:
我收到两种类型的消息,具体取决于我尝试连接的服务器:
root@hello-node-7bf657c596-smr2h:/# openssl s_client -connect google.com:443 -debug
CONNECTED(00000003)
write to 0xda15b0 [0xda1630] (305 bytes => 305 (0x131))
0000 - 16 03 01 01 2c 01 00 01-28 03 03 4e c0 cf d9 62 ....,...(..N...b
0010 - 58 6e 88 7b 28 7f e8 c1-62 c6 8f 23 60 86 b0 53 Xn.{(...b..#`..S
0020 - df 43 cd a7 58 52 bc 59-6a ae bf 00 00 aa c0 30 .C..XR.Yj......0
0030 - c0 2c c0 28 c0 24 c0 14-c0 0a 00 a5 00 a3 00 a1 .,.(.$..........
0040 - 00 9f 00 6b 00 6a 00 69-00 68 00 39 00 38 00 37 ...k.j.i.h.9.8.7
0050 - 00 36 00 88 00 87 00 86-00 85 c0 32 c0 2e c0 2a .6.........2...*
0060 - c0 26 c0 0f c0 05 00 9d-00 3d 00 35 00 84 c0 2f .&.......=.5.../
0070 - c0 2b c0 27 c0 23 c0 13-c0 09 00 a4 00 a2 00 a0 .+.'.#..........
0080 - 00 9e 00 67 00 40 00 3f-00 3e 00 33 00 32 00 31 ...g.@.?.>.3.2.1
0090 - 00 30 00 9a 00 99 00 98-00 97 00 45 00 44 00 43 .0.........E.D.C
00a0 - 00 42 c0 31 c0 2d c0 29-c0 25 c0 0e c0 04 00 9c .B.1.-.).%......
00b0 - 00 3c 00 2f 00 96 00 41-c0 11 c0 07 c0 0c c0 02 .<./...A........
00c0 - 00 05 00 04 c0 12 c0 08-00 16 00 13 00 10 00 0d ................
00d0 - c0 0d c0 03 00 0a 00 ff-01 00 00 55 00 0b 00 04 ...........U....
00e0 - 03 00 01 02 00 0a 00 1c-00 1a 00 17 00 19 00 1c ................
00f0 - 00 1b 00 18 00 1a 00 16-00 0e 00 0d 00 0b 00 0c ................
0100 - 00 09 00 0a 00 23 00 00-00 0d 00 20 00 1e 06 01 .....#..... ....
0110 - 06 02 06 03 05 01 05 02-05 03 04 01 04 02 04 03 ................
0120 - 03 01 03 02 03 03 02 01-02 02 02 03 00 0f 00 01 ................
0130 - 01 .
read from 0xda15b0 [0xda6b90] (7 bytes => 7 (0x7))
0000 - 15 00 00 00 02 02 28 ......(
140080050284184:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:794:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 305 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1600154660
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
和
root@hello-node-7bf657c596-smr2h:/# openssl s_client -connect pypi.org:443 -debug
CONNECTED(00000003)
write to 0x24645b0 [0x2464630] (305 bytes => 305 (0x131))
0000 - 16 03 01 01 2c 01 00 01-28 03 03 0c 2b bf dc 71 ....,...(...+..q
0010 - d2 41 56 f3 40 e4 c5 3f-62 88 cf 2a a7 76 e4 a4 .AV.@..?b..*.v..
0020 - 08 96 13 89 30 13 fa 75-c9 2d 32 00 00 aa c0 30 ....0..u.-2....0
0030 - c0 2c c0 28 c0 24 c0 14-c0 0a 00 a5 00 a3 00 a1 .,.(.$..........
0040 - 00 9f 00 6b 00 6a 00 69-00 68 00 39 00 38 00 37 ...k.j.i.h.9.8.7
0050 - 00 36 00 88 00 87 00 86-00 85 c0 32 c0 2e c0 2a .6.........2...*
0060 - c0 26 c0 0f c0 05 00 9d-00 3d 00 35 00 84 c0 2f .&.......=.5.../
0070 - c0 2b c0 27 c0 23 c0 13-c0 09 00 a4 00 a2 00 a0 .+.'.#..........
0080 - 00 9e 00 67 00 40 00 3f-00 3e 00 33 00 32 00 31 ...g.@.?.>.3.2.1
0090 - 00 30 00 9a 00 99 00 98-00 97 00 45 00 44 00 43 .0.........E.D.C
00a0 - 00 42 c0 31 c0 2d c0 29-c0 25 c0 0e c0 04 00 9c .B.1.-.).%......
00b0 - 00 3c 00 2f 00 96 00 41-c0 11 c0 07 c0 0c c0 02 .<./...A........
00c0 - 00 05 00 04 c0 12 c0 08-00 16 00 13 00 10 00 0d ................
00d0 - c0 0d c0 03 00 0a 00 ff-01 00 00 55 00 0b 00 04 ...........U....
00e0 - 03 00 01 02 00 0a 00 1c-00 1a 00 17 00 19 00 1c ................
00f0 - 00 1b 00 18 00 1a 00 16-00 0e 00 0d 00 0b 00 0c ................
0100 - 00 09 00 0a 00 23 00 00-00 0d 00 20 00 1e 06 01 .....#..... ....
0110 - 06 02 06 03 05 01 05 02-05 03 04 01 04 02 04 03 ................
0120 - 03 01 03 02 03 03 02 01-02 02 02 03 00 0f 00 01 ................
0130 - 01 .
read from 0x24645b0 [0x2469b90] (7 bytes => 7 (0x7))
0000 - 15 03 03 00 02 02 28 ......(
140284912154264:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:769:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 305 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1600154752
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
我发现这两条消息一起出现,如下所述:https://blog.techstacks.com/2010/03/3-common-causes-of-unknown-ssl-protocol-errors-with-curl.html
目标站点不喜欢该密码 您可能尝试使用站点配置为拒绝的 SSL 密码连接到站点。例如,面向客户的 SSL 加密站点通常会禁用匿名密码。(我们中的许多人对任何 SSL 加密的网站都设置了全面的拒绝策略——无论其用途如何。)以下命令字符串“可能”也会导致 curl (35) 错误:
curl --ciphers ADH-RC4-MD5 https://some_web_site.some_domain.com/ 不幸的是,您可以从 curl 获得的错误响应类型在很大程度上取决于 SSL 服务器。在某些网站上,您会收到未知 SSL 协议错误,但在我的 techstacks-tools 网站上,我收到:
curl:(35)错误:14077410:SSL 例程:SSL23_GET_SERVER_HELLO:sslv3 警报握手失败
我发现的另一件事是,我在 DigitalOcean droplet 上准备了相同的环境,并且那里没有任何问题。
我的裸机环境对外界不可用,它位于内部网络内 - 也许这有某种关联?
我也尝试使用 microk8s 设置 k8s,但问题仍然存在,所以这可能与 rancher 无关,而仅与我的机器设置有关。
Pod 的 YAML:
apiVersion: v1
kind: Pod
metadata:
annotations:
cni.projectcalico.org/podIP: 10.42.0.8/32
cni.projectcalico.org/podIPs: 10.42.0.8/32
creationTimestamp: "2020-09-14T19:51:57Z"
generateName: hello-node-7bf657c596-
labels:
app: hello-node
pod-template-hash: 7bf657c596
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:generateName: {}
f:labels:
.: {}
f:app: {}
f:pod-template-hash: {}
f:ownerReferences:
.: {}
k:{"uid":"dda770d0-3aa3-4fc9-97e1-c929fc3629e9"}:
.: {}
f:apiVersion: {}
f:blockOwnerDeletion: {}
f:controller: {}
f:kind: {}
f:name: {}
f:uid: {}
f:spec:
f:containers:
k:{"name":"echoserver"}:
.: {}
f:image: {}
f:imagePullPolicy: {}
f:name: {}
f:resources: {}
f:terminationMessagePath: {}
f:terminationMessagePolicy: {}
f:dnsPolicy: {}
f:enableServiceLinks: {}
f:restartPolicy: {}
f:schedulerName: {}
f:securityContext: {}
f:terminationGracePeriodSeconds: {}
manager: kube-controller-manager
operation: Update
time: "2020-09-14T19:51:57Z"
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:status:
f:conditions:
.: {}
k:{"type":"PodScheduled"}:
.: {}
f:lastProbeTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
manager: kube-scheduler
operation: Update
time: "2020-09-14T19:51:57Z"
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.: {}
f:cni.projectcalico.org/podIP: {}
f:cni.projectcalico.org/podIPs: {}
manager: calico
operation: Update
time: "2020-09-14T19:52:01Z"
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:status:
f:conditions:
k:{"type":"ContainersReady"}:
.: {}
f:lastProbeTime: {}
f:lastTransitionTime: {}
f:status: {}
f:type: {}
k:{"type":"Initialized"}:
.: {}
f:lastProbeTime: {}
f:lastTransitionTime: {}
f:status: {}
f:type: {}
k:{"type":"Ready"}:
.: {}
f:lastProbeTime: {}
f:lastTransitionTime: {}
f:status: {}
f:type: {}
f:containerStatuses: {}
f:hostIP: {}
f:phase: {}
f:podIP: {}
f:podIPs:
.: {}
k:{"ip":"10.42.0.8"}:
.: {}
f:ip: {}
f:startTime: {}
manager: kubelet
operation: Update
time: "2020-09-14T19:52:01Z"
name: hello-node-7bf657c596-smr2h
namespace: default
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: ReplicaSet
name: hello-node-7bf657c596
uid: dda770d0-3aa3-4fc9-97e1-c929fc3629e9
resourceVersion: "3468"
selfLink: /api/v1/namespaces/default/pods/hello-node-7bf657c596-smr2h
uid: cf37b215-8269-42ce-9e70-750f9f862cac
spec:
containers:
- image: k8s.gcr.io/echoserver:1.4
imagePullPolicy: IfNotPresent
name: echoserver
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-9jlms
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: gepard
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: default-token-9jlms
secret:
defaultMode: 420
secretName: default-token-9jlms
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2020-09-14T19:52:00Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2020-09-14T19:52:01Z"
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2020-09-14T19:52:01Z"
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2020-09-14T19:52:00Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: docker://2d2ddbf42fc4e6634a782c7036cf4a9a1d9f50a3d847bb5444932c701cd8186e
image: k8s.gcr.io/echoserver:1.4
imageID: docker-pullable://k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb
lastState: {}
name: echoserver
ready: true
restartCount: 0
started: true
state:
running:
startedAt: "2020-09-14T19:52:01Z"
hostIP: 10.0.0.3
phase: Running
podIP: 10.42.0.8
podIPs:
- ip: 10.42.0.8
qosClass: BestEffort
startTime: "2020-09-14T19:52:00Z"
防火墙目前已被禁用。
答案1
OP 选择不发布答案,但确实说他们解决了这个问题:
问题是
/etc/resolve.conf
主机的搜索部分添加了一些奇怪的域名。当我尝试发出 https 请求时,DNS 已将我重定向到其他使用纯 http 响应的服务器。纠正 DNS 设置解决了该问题。