我有一个配置了 calico 作为网络系统的 k8s 集群。我正在运行稍微定制的 helm elastic/elasticsearch 和 elastic/kibana 版本，并启用了安全性。为了安全起见，我使用 Let's Encrypt 证书。启动 kibana 时，与 elasticsearch 实例的连接失败并出现此错误
{"type":"log","@timestamp":"2021-06-01T13:09:55+00:00","tags":["debug","elasticsearch","query","data"],"pid":952,"message":"[ConnectionError]: unable to get issuer certificate"}
我可以通过在 kibana 中禁用证书验证来解决这个问题。有人能看出它为什么会失败吗？
密钥库创建
cat cert1.pem > store.pem
cat privkey1.pem >> store.pem
cat chain1.pem >> store.pem
cat fullchain1.pem >> store.pem
openssl pkcs12 -export -in store.pem -out keystore.pkcs12
值_elastic.yaml
replicas: 1
minimumMasterNodes: 1
esConfig:
elasticsearch.yml: |
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
xpack.security.enabled: true
extraEnvs:
- name: ELASTIC_PASSWORD
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password
- name: ELASTIC_USERNAME
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username
secretMounts:
- name: elastic-certificates
secretName: elastic-certificates
path: /usr/share/elasticsearch/config/certs-gen/
protocol: https
service:
labels: {}
labelsHeadless: {}
type: NodePort
nodePort: 30001
annotations: {}
httpPortName: http
transportPortName: transport
loadBalancerIP: ""
loadBalancerSourceRanges: []
externalTrafficPolicy: ""
clusterHealthCheckParams: "wait_for_status=yellow&timeout=1s"
值_kibana.yaml
elasticsearchHosts: "redacted its a TLD with appropriate port"
extraEnvs:
- name: "NODE_OPTIONS"
value: "--max-old-space-size=1800"
- name: 'ELASTICSEARCH_USERNAME'
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username
- name: 'ELASTICSEARCH_PASSWORD'
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password
- name: 'KIBANA_ENCRYPTION_KEY'
valueFrom:
secretKeyRef:
name: kibana
key: encryptionkey
- name: 'LOGGING_VERBOSE'
value: "true"
secretMounts:
- name: elastic-certificates
secretName: elastic-certificates
path: /usr/share/kibana/config/certs-gen/
kibanaConfig:
kibana.yml: |
server.ssl:
enabled: true
key: /usr/share/kibana/config/certs-gen/privkey1.pem
certificate: /usr/share/kibana/config/certs-gen/fullchain1.pem
elasticsearch.ssl:
certificateAuthorities: /usr/share/kibana/config/certs-gen/fullchain1.pem
verificationMode: certificate
xpack.reporting.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
xpack.security.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
xpack.encryptedSavedObjects.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
protocol: https
service:
type: NodePort
loadBalancerIP: ""
port: 5601
nodePort: 30002
labels: {}
annotations: {}
# cloud.google.com/load-balancer-type: "Internal"
# service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
# service.beta.kubernetes.io/azure-load-balancer-internal: "true"
# service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
# service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
loadBalancerSourceRanges: []
# 0.0.0.0/0
httpPortName: HTTP
kubectl 获取 pv、pvc、节点、pod、svc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/elk-data 30Gi RWO Retain Bound default/elasticsearch-master-elasticsearch-master-0 40m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/elasticsearch-master-elasticsearch-master-0 Bound elk-data 30Gi RWO 32m
NAME STATUS ROLES AGE VERSION
node/kubeloadbalan Ready control-plane,master 28h v1.21.1
NAME READY STATUS RESTARTS AGE
pod/elasticsearch-master-0 1/1 Running 0 13m
pod/kibana-kibana-7fdbd7c66d-bg5xb 0/1 Running 0 7m1s
pod/nginx-deployment-868c6bb874-tsbg4 1/1 Running 0 40m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elasticsearch-master NodePort 10.97.47.66 <none> 9200:30001/TCP,9300:32185/TCP 13m
service/elasticsearch-master-headless ClusterIP None <none> 9200/TCP,9300/TCP 13m
service/kibana-kibana NodePort 10.101.78.90 <none> 5601:30002/TCP 7m1s
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 28h
service/nginx-service NodePort 10.100.231.27 <none> 80:30000/TCP 40m
从容器内部：
kubectl exec pod/kibana-kibana-7fdbd7c66d-bg5xb -it bash
curl -k -u redacted:redacted https://redacted:30001
{
"name" : "elasticsearch-master-0",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "iXEuqB7iQ9abptIZ_Gp1yg",
"version" : {
"number" : "7.13.0",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "5ca8591c6fcdb1260ce95b08a8e023559635c6f3",
"build_date" : "2021-05-19T22:22:26.081971330Z",
"build_snapshot" : false,
"lucene_version" : "8.8.2",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
curl -u redacted:redacted https://redacted:30001
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
这表明，即使 curl 也无法从容器内部验证合法性。这可能表明问题也出在 elasticsearch 上。由于冗长的内容增加，完整的 Kibana 日志一团糟。我可以根据要求发布它。
我为此投入了好几天的时间。我精疲力竭，不知道该如何继续。
答案1
通过在评论中讨论问题的可能根本原因（缺少 CA），我自己找到了解决方案。该问题是由于添加了错误的 CA 引起的
elasticsearch.ssl:
certificateAuthorities: /usr/share/kibana/config/certs-gen/fullchain1.pem
verificationMode: certificate
删除这些条目即可解决问题。