我在 Azure 上有一个 kubernetes，它在 V1.22 上运行良好，我对其进行了升级，但在构建平台服务（使用 chart helm）时遇到了问题。
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
terraform apply dev.planfile
╷
│ Warning: "use_microsoft_graph": [DEPRECATED] This field now defaults to `true` and will be removed in v1.3 of Terraform Core due to the deprecation of ADAL by Microsoft.
│
│
╵
Acquiring state lock. This may take a few moments...
null_resource.restart[0]: Destroying... [id=4241487241670744054]
null_resource.restart[0]: Destruction complete after 0s
local_file.kubeconfig: Creating...
local_file.kubeconfig: Creation complete after 0s [id=520d2cb412c868b51d8ae0eb84e1dd974b7f8d06]
helm_release.timeseries_management_service: Destroying... [id=srv-management-timeseries]
helm_release.databases_management_service: Destroying... [id=srv-management-databases]
helm_release.worksites_management_service: Destroying... [id=srv-management-worksites]
helm_release.srv_vendor_capetti: Modifying... [id=srv-vendor-capetti]
helm_release.srv_vendor_sisgeo: Modifying... [id=srv-vendor-sisgeo]
helm_release.srv_vendor_ellona: Modifying... [id=srv-vendor-ellona]
helm_release.srv_vendor_senceive: Modifying... [id=srv-vendor-senceive]
helm_release.weather_srv: Modifying... [id=weather-srv]
helm_release.srv_vendor_ftp: Modifying... [id=srv-vendor-ftp]
helm_release.srv_vendor_sites: Modifying... [id=srv-vendor-sites]
helm_release.srv_vendor_geokon: Modifying... [id=srv-vendor-geokon]
helm_release.srv_vendor_worldsensing: Modifying... [id=srv-vendor-worldsensing]
null_resource.restart[0]: Creating...
null_resource.restart[0]: Provisioning with 'local-exec'...
null_resource.restart[0] (local-exec): Executing: ["/bin/sh" "-c" "kubectl --kubeconfig=./.kube/aks-dev get po -n platform | tail -n +2 | awk '{print $1}' | xargs kubectl --kubeconfig=./.kube/aks-dev delete pod -n platform"]
helm_release.srv_monitoring_rabbitmq: Modifying... [id=srv-monitoring-rabbitmq]
helm_release.srv_vendor_gantner: Modifying... [id=srv-vendor-gantner]
helm_release.users_management_service: Modifying... [id=srv-management-users]
helm_release.srv_vendor_campbell: Modifying... [id=srv-vendor-campbell]
null_resource.restart[0] (local-exec): pod "behavior-tewapp-766b876bf5-zhctb" deleted
null_resource.restart[0] (local-exec): pod "blobstorage-tewapp-6c67f47489-jlrbf" deleted
helm_release.worksites_management_service: Destruction complete after 4s
null_resource.restart[0] (local-exec): pod "cmms-tewapp-59888fb86c-rjpj7" deleted
helm_release.timeseries_management_service: Destruction complete after 4s
null_resource.restart[0] (local-exec): pod "connectivity-totalstation-tewapp-5b6684588-787dv" deleted
helm_release.databases_management_service: Destruction complete after 4s
null_resource.restart[0] (local-exec): pod "dataintegrator-tewapp-67fff5f5b7-9c5kl" deleted
null_resource.restart[0] (local-exec): pod "dataintegrator-tewapp-67fff5f5b7-q2jk8" deleted
null_resource.restart[0] (local-exec): pod "jag3d-tewapp-cd5fb67dd-8lcf7" deleted
null_resource.restart[0] (local-exec): pod "platform-api-tewapp-68fd65679f-btb8h" deleted
null_resource.restart[0] (local-exec): pod "portal-v2-tewapp-fd85874c6-s2xhf" deleted
null_resource.restart[0] (local-exec): pod "progression-tewapp-f45c77644-ntt9m" deleted
null_resource.restart[0] (local-exec): pod "reprocess-tewapp-59f897b887-c22vm" deleted
null_resource.restart[0] (local-exec): pod "srv-config-toponode-tewapp-759895bf56-cn846" deleted
null_resource.restart[0] (local-exec): pod "srv-data-compute-tewapp-6b65cc66c8-c4sp7" deleted
null_resource.restart[0] (local-exec): pod "srv-data-toponode-tewapp-5d5cf54dd4-2sdqx" deleted
null_resource.restart[0] (local-exec): pod "srv-lora-tewapp-7bf8899c8-jlhvr" deleted
null_resource.restart[0] (local-exec): pod "srv-lora-vendor-a3ip-tewapp-68b56f554d-st45c" deleted
null_resource.restart[0] (local-exec): pod "srv-lora-vendor-ineosense-tewapp-6f68ffd77b-flphn" deleted
null_resource.restart[0] (local-exec): pod "srv-lora-vendor-mcf88-tewapp-5ddf549d54-sb6l6" deleted
null_resource.restart[0] (local-exec): pod "srv-management-users-tewapp-79597d4b7c-pg5th" deleted
null_resource.restart[0] (local-exec): pod "srv-monitoring-rabbitmq-28311030-vzt9v" deleted
null_resource.restart[0] (local-exec): pod "srv-monitoring-rabbitmq-28311060-bjwd6" deleted
null_resource.restart[0] (local-exec): pod "srv-monitoring-rabbitmq-28311090-2clcc" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-a3ip-tewapp-bc8f5fd8b-c5zdd" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-ackcio-tewapp-845b8b9664-cvk98" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-capetti-28310940-v7dlc" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-capetti-28311000-2wplx" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-capetti-28311060-bcwn8" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-ellona-28311075-7trtl" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-ellona-28311090-pghkt" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-ellona-28311105-jd8zq" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-geokon-28311030-4lhkd" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-geokon-28311060-p7z9q" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-geokon-28311090-2mfvr" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-gss-tewapp-86899b99fb-7w2xq" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-mcf88-tewapp-7887f5bd4d-gmvvv" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-senceive-28311030-4p2g4" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-senceive-28311060-jsbpc" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-senceive-28311090-m7xwv" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-sisgeo-28311030-ws2np" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-sisgeo-28311060-k6w88" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-sisgeo-28311090-vfp5r" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-sites-28311045-pktkq" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-sites-28311075-745g8" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-sites-28311105-hph7j" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-worldsensing-28311030-pxsrl" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-worldsensing-28311060-hm5qf" deleted
null_resource.restart[0] (local-exec): pod "srv-vendor-worldsensing-28311090-rjh68" deleted
null_resource.restart[0] (local-exec): pod "tewadmin-api-tewapp-78bd6db7b5-vnjs2" deleted
null_resource.restart[0] (local-exec): pod "tewapi-data-tewapp-846986f687-br8mm" deleted
null_resource.restart[0] (local-exec): pod "tewapi-tewapp-6db7d5f867-zrnsk" deleted
null_resource.restart[0]: Still creating... [10s elapsed]
null_resource.restart[0] (local-exec): pod "toponode-data-tewapp-ff6f586c5-jc7mv" deleted
helm_release.users_management_service: Still modifying... [id=srv-management-users, 10s elapsed]
null_resource.restart[0] (local-exec): pod "totalstation-acceptor-tewapp-57d76c5c4b-7g98v" deleted
null_resource.restart[0] (local-exec): pod "totalstation-mesh-tewapp-694fcc4fc7-wr9qj" deleted
null_resource.restart[0] (local-exec): pod "uby-data-api-tewapp-7bbbb95c46-ngzrt" deleted
null_resource.restart[0] (local-exec): pod "weather-srv-28310760-c29xm" deleted
null_resource.restart[0] (local-exec): pod "weather-srv-28310880-vf4hh" deleted
null_resource.restart[0] (local-exec): pod "weather-srv-28311000-shn9j" deleted
null_resource.restart[0] (local-exec): Error from server (NotFound): pods "srv-management-databases-tewapp-7df58b9979-ps8hl" not found
null_resource.restart[0] (local-exec): Error from server (NotFound): pods "srv-management-timeseries-tewapp-7dfd849557-wmh7j" not found
null_resource.restart[0] (local-exec): Error from server (NotFound): pods "srv-management-worksites-tewapp-5f545c9d66-hfkdn" not found
helm_release.users_management_service: Modifications complete after 17s [id=srv-management-users]
╷
│ Error: local-exec provisioner error
│
│ with null_resource.restart[0],
│ on restart-pods.tf line 24, in resource "null_resource" "restart":
│ 24: provisioner "local-exec" {
│
│ Error running command 'kubectl --kubeconfig=./.kube/aks-dev get po -n
│ platform | tail -n +2 | awk '{print $1}' | xargs kubectl
│ --kubeconfig=./.kube/aks-dev delete pod -n platform': exit status 123.
│ Output: pod "behavior-tewapp-766b876bf5-zhctb" deleted
│ pod "blobstorage-tewapp-6c67f47489-jlrbf" deleted
│ ...
│ pod "weather-srv-28311000-shn9j" deleted
│ Error from server (NotFound): pods
│ "srv-management-databases-tewapp-7df58b9979-ps8hl" not found
│ Error from server (NotFound): pods
│ "srv-management-timeseries-tewapp-7dfd849557-wmh7j" not found
│ Error from server (NotFound): pods
│ "srv-management-worksites-tewapp-5f545c9d66-hfkdn" not found
│
╵
╷
│ Error: unable to build kubernetes objects from current release manifest: resource mapping not found for name: "srv-monitoring-rabbitmq" namespace: "platform" from "": no matches for kind "CronJob" in version "batch/v1beta1"
│ ensure CRDs are installed first
│
│ with helm_release.srv_monitoring_rabbitmq,
│ on srv-monitoring-rabbitmq.tf line 1, in resource "helm_release" "srv_monitoring_rabbitmq":
│ 1: resource "helm_release" "srv_monitoring_rabbitmq" {
│
╵
╷
│ Error: unable to build kubernetes objects from current release manifest: resource mapping not found for name: "srv-vendor-campbell" namespace: "platform" from "": no matches for kind "CronJob" in version "batch/v1beta1"
│ ensure CRDs are installed first
│
│ with helm_release.srv_vendor_campbell,
│ on srv-vendor-campbell.tf line 1, in resource "helm_release" "srv_vendor_campbell":
│ 1: resource "helm_release" "srv_vendor_campbell" {
│
╵
╷
│ Error: unable to build kubernetes objects from current release manifest: resource mapping not found for name: "srv-vendor-capetti" namespace: "platform" from "": no matches for kind "CronJob" in version "batch/v1beta1"
│ ensure CRDs are installed first
│
╵...
Releasing state lock. This may take a few moments...
make: *** [Makefile:10: apply] Error 1
Cleaning up project directory and file based variables 00:00
ERROR: Job failed: exit code 2
我尝试改变：
Job/Chart.yaml 中的“Versions”和“Versions”值参数“batch/v1beta1”中的“apiVersion”更改为“batch/v1”tewjob/templates/cronjob.yaml 中的“source = “hashicorp/helm””的“Version”“~> 2.6.0”更改为“~> 3.13.1”在providers.tf中但没有任何变化（我的图表被Bitnami超载）。你有什么想法吗？
我有许多 terraform 配置文件，我已截断日志。对于 Rabbit，它仅用于监控它。例如，对于服务：
resource "helm_release" "behavior" {
name = "behavior"
namespace = kubernetes_namespace.platform.metadata.0.name
chart = local.tewapp_chart_path
set {
name = "chart_hash"
value = local.tewapp_chart_hash
}
values = [file("behavior-${local.env}.yaml"), <<-EOT
extraEnvVars:
- name: "ConnectionStrings__MSSQL"
value: "Data Source=mssql1.data;Initial Catalog=testdb;User=${data.terraform_remote_state.data.outputs.mssql1_user};Password=${data.terraform_remote_state.data.outputs.mssql1_pass}"
- name: "ConnectionStrings__rabbitmq"
value: "amqp://${data.terraform_remote_state.data.outputs.rabbitmq_user}:${data.terraform_remote_state.data.outputs.rabbitmq_admin_pass}@rabbitmq.data"
resources:
requests:
cpu: 5m
memory: 501Mi
limits:
cpu: 100m
memory: 1000Mi
EOT
]
set {
name = "image.tag"
value = local.behavior_tag
}
set {
name = "name"
value = "behavior"
}
}
问候
答案1
这是因为我在重定向 API 之前更新了 kube 版本。为了纠正它，我必须删除所有 cronJobs 并重新创建它们：
- kubectl 获取 secret -n 平台 | awk '/srv-vendor-MyJob1.v/{print $1}' | xargs kubectl 删除 secret -n 平台
- kubectl 删除-n 平台 cronjob srv-vendor-MyJob1
适用于所有工作
并重建