我们已将 vault 配置为集群中的 pod 运行。在下面的部署 YAML 文件中，我们已包含 pod 首次启动时发生的 vault 初始化和解封。但是当 pod 重新启动时，pod 将进入 crashLoopBackOff 状态，因为 vault 正在重新初始化。这是因为我们在部署文件的 postStart 生命周期命令中同时包含了初始化和解封命令。有没有什么方法可以让我们只初始化 pod 一次，然后在 pod 重新启动时使用现有密钥解封 vault？
部署文件：
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: vault
name: vault
spec:
replicas: 1
template:
metadata:
labels:
app: vault
spec:
containers:
- image: vault
name: vault
imagePullPolicy: Always
ports:
- containerPort: 8200
name: vaultport
protocol: TCP
securityContext:
capabilities:
add:
- IPC_LOCK
env:
- name: VAULT_ADDR
value: "http://0.0.0.0:8200"
command: ["vault", "server"]
args:
- "-config=/vault/config/config.hcl"
volumeMounts:
- name: vault-unseal
mountPath: /vault/file/unseal.sh
subPath: unseal.sh
- name: vault-config
mountPath: /vault/config/config.hcl
subPath: config.hcl
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "vault operator init > /vault/file/keys.txt; sh /vault/file/unseal.sh" ]
volumes:
- name: vault-unseal
configMap:
name: vault-unseal
- name: vault-config
configMap:
name: vault-config
imagePullSecrets:
- name: regcred
kubectl describe pod 的输出：
Name: vault-677bfd9c9c-dwsgv
Namespace: xxx
Priority: 0
Node: xxxxxxx-5b587f98-ljf4/10.0.0.11
Start Time: Thu, 30 Jan 2020 06:26:21 +0000
Labels: app=vault
pod-template-hash=677bfd9c9c
Annotations: <none>
Status: Running
IP: 10.4.2.10
IPs: <none>
Controlled By: ReplicaSet/vault-677bfd9c9c
Containers:
vault:
Container ID: xxxxxxxxxxx
Image: xxxxxxxxxxxxxxxx
Image ID: xxxxxxxxxxxxxxxxxxxxxxxxx
Port: 8200/TCP
Host Port: 0/TCP
Command:
vault
server
Args:
-config=/vault/config/config.hcl
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Completed
Exit Code: 0
Started: Thu, 30 Jan 2020 06:26:26 +0000
Finished: Thu, 30 Jan 2020 06:26:27 +0000
Ready: False
Restart Count: 1
Environment:
VAULT_ADDR: http://0.0.0.0:8200
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-kxfdb (ro)
/vault/config/config.hcl from vault-config (rw,path="config.hcl")
/vault/file from vault-data (rw)
/vault/file/unseal.sh from vault-unseal (rw,path="unseal.sh")
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
vault-unseal:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: vault-unseal
Optional: false
vault-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: vault-config
Optional: false
vault-data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: vault-data
ReadOnly: false
default-token-kxfdb:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-kxfdb
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 18s default-scheduler Successfully assigned xxx/xxxxxxxxxx
Normal Pulling 13s (x2 over 15s) kubelet, gke-cluster-testing--np-testing-featu-5b587f98-ljf4 pulling image "xxxxxxxxx"
Normal Pulled 13s (x2 over 15s) kubelet, gke-cluster-testing--np-testing-featu-5b587f98-ljf4 Successfully pulled image "xxxxxxx"
Normal Created 13s (x2 over 15s) kubelet, gke-cluster-testing--np-testing-featu-5b587f98-ljf4 Created container
Normal Started 13s (x2 over 14s) kubelet, gke-cluster-testing--np-testing-featu-5b587f98-ljf4 Started container
Warning FailedPostStartHook 12s (x2 over 14s) kubelet, gke-cluster-testing--np-testing-featu-5b587f98-ljf4 Exec lifecycle hook ([/bin/sh -c vault operator init > /vault/file/keys.txt; sh /vault/file/unseal.sh]) for Container "vault" in Pod "vault-677bfd9c9c-dwsgv_xxx(6ebdc17a-4329-11ea-9fc1-4201c0a80004)" failed - error: command '/bin/sh -c vault operator init > /vault/file/keys.txt; sh /vault/file/unseal.sh' exited with 2: Error initializing: Error making API request.
URL: PUT http://0.0.0.0:8200/v1/sys/init
Code: 400. Errors:
* Vault is already initialized
An error occurred attempting to ask for an unseal key. The raw error message
is shown below, but usually this is because you attempted to pipe a value
into the unseal command or you are executing outside of a terminal (tty). You
should run the unseal command from a terminal for maximum security. If this
is not an option, the unseal key can be provided as the first argument to the
unseal command. The raw error was: file descriptor 0 is not a terminal
An error occurred attempting to ask for an unseal key. The raw error message
is shown below, but usually this is because you attempted to pipe a value
into the unseal command or you are executing outside of a terminal (tty). You
should run the unseal command from a terminal for maximum security. If this
is not an option, the unseal key can be provided as the first argument to the
unseal command. The raw error was: file descriptor 0 is not a terminal
An error occurred attempting to ask for an unseal key. The raw error message
is shown below, but usually this is because you attempted to pipe a value
into the unseal command or you are executing outside of a terminal (tty). You
should run the unseal command from a terminal for maximum security. If this
is not an option, the unseal key can be provided as the first argument to the
unseal command. The raw error was: file descriptor 0 is not a terminal
Token (will be hidden):
Error authenticating: An error occurred attempting to ask for a token. The raw error message is shown below, but usually this is because you attempted to pipe a value into the command or you are executing outside of a terminal (tty). If you want to pipe the value, pass "-" as the argument to read from stdin. The raw error was: file descriptor 0 is not a terminal
, message: "Unseal Key (will be hidden): \nUnseal Key (will be hidden): \nUnseal Key (will be hidden): \nKey Value\n--- -----\nSeal Type shamir\nInitialized true\nSealed true\nTotal Shares 5\nThreshold 3\nUnseal Progress 0/3\nUnseal Nonce n/a\nVersion 1.3.2\nHA Enabled false\n++++++++++++ Vault Status +++++++++\nKey Value\n--- -----\nSeal Type shamir\nInitialized true\nSealed true\nTotal Shares 5\nThreshold 3\nUnseal Progress 0/3\nUnseal Nonce n/a\nVersion 1.3.2\nHA Enabled false\nError initializing: Error making API request.\n\nURL: PUT http://0.0.0.0:8200/v1/sys/init\nCode: 400. Errors:\n\n* Vault is already initialized\nAn error occurred attempting to ask for an unseal key. The raw error message\nis shown below, but usually this is because you attempted to pipe a value\ninto the unseal command or you are executing outside of a terminal (tty). You\nshould run the unseal command from a terminal for maximum security. If this\nis not an option, the unseal key can be provided as the first argument to the\nunseal command. The raw error was: file descriptor 0 is not a terminal\nAn error occurred attempting to ask for an unseal key. The raw error message\nis shown below, but usually this is because you attempted to pipe a value\ninto the unseal command or you are executing outside of a terminal (tty). You\nshould run the unseal command from a terminal for maximum security. If this\nis not an option, the unseal key can be provided as the first argument to the\nunseal command. The raw error was: file descriptor 0 is not a terminal\nAn error occurred attempting to ask for an unseal key. The raw error message\nis shown below, but usually this is because you attempted to pipe a value\ninto the unseal command or you are executing outside of a terminal (tty). You\nshould run the unseal command from a terminal for maximum security. If this\nis not an option, the unseal key can be provided as the first argument to the\nunseal command. The raw error was: file descriptor 0 is not a terminal\nToken (will be hidden): \nError authenticating: An error occurred attempting to ask for a token. The raw error message is shown below, but usually this is because you attempted to pipe a value into the command or you are executing outside of a terminal (tty). If you want to pipe the value, pass \"-\" as the argument to read from stdin. The raw error was: file descriptor 0 is not a terminal\n"
Normal Killing 12s (x2 over 14s) kubelet, gke-cluster-testing--np-testing-featu-5b587f98-ljf4 Killing container with id docker://vault:FailedPostStartHook
Warning BackOff 10s (x2 over 11s) kubelet, gke-cluster-testing--np-testing-featu-5b587f98-ljf4 Back-off restarting failed container
答案1
是的，在这种情况下，您需要让您的脚本更智能，首先检查保险库是否已初始化。这是一个非常简单的 bash 脚本示例。
此外，实现此目的的常见模式是使用 sidecar（最终无限期地进入“睡眠”状态），但postStart只要你的脚本具有足够的弹性，钩子也应该可以工作，因为在某些情况下它会在 vault 容器启动之前运行，最终会杀死它并重新启动，直到它们最终同步。
我强烈推荐Seth Vargo 的演讲自动初始化由更复杂的 Golang 程序完成。