☰

k3s 从属代理未加入 kubernetes 集群（在 Vagrantbox 设置中）

tag-icon • tag-icon networking virtualbox ssl vagrant kubernetes
k3s 从属代理未加入 kubernetes 集群（在 Vagrantbox 设置中）

出于学习目的，我尝试使用 Vagrant 在 Virtualbox VM 中设置 k3s 集群。但我的从属代理未连接到我的主节点。我的 VM 提供商是 Virtualbox。

我正在按照快速入门指南中的说明进行操作 ->https://rancher.com/docs/k3s/latest/en/quick-start

主节点设置(script = provision/master-node.sh) ... 我将令牌复制到 Vagrant 文件夹，以便所有其他 Vagrantbox 都可以使用它。它们共享相同的 Vagrantfile。调用脚本时，令牌文件存在于所有节点中（我已检查）：

k3sTokenFile="/var/lib/rancher/k3s/server/node-token"

echo "[INFO] Install k3s on master-node"
curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" K3S_NODE_NAME="$HOSTNAME" sh -
echo "[INFO] Get K3S_TOKEN from master-node"
cp "$k3sTokenFile" /vagrant/resources/.generated/k3s.token

主节点设置自行运行。我可以部署应用程序并通过浏览器访问（我部署了仪表板应用程序）。

从节点设置(script = provision/worker-node.sh) ... 主节点的 IP 是正确的（请参阅文章末尾的 Vagrantfile）：

echo "[INFO] Read K3S_TOKEN from filesystem"
tmp=$(</vagrant/resources/.generated/k3s.token)
k3sToken=${tmp%%*( )} # trim witespaces

masterIP="192.168.30.10"

echo "[INFO] Install k3s on worker-node and join cluster"
curl -sfL https://get.k3s.io | K3S_URL="https://$masterIP:6443" K3S_TOKEN="$k3sToken" K3S_NODE_NAME="$HOSTNAME" sh -

这是从属代理的日志输出设置：

v-k3s-worker-3: [INFO] Install k3s on worker-node and join cluster
v-k3s-worker-3: [INFO]  Finding release for channel stable
v-k3s-worker-3: [INFO]  Using v1.21.3+k3s1 as release
v-k3s-worker-3: [INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.21.3+k3s1/sha256sum-amd64.txt
v-k3s-worker-3: [INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.21.3+k3s1/k3s
v-k3s-worker-3: [INFO]  Verifying binary download
v-k3s-worker-3: [INFO]  Installing k3s to /usr/local/bin/k3s
v-k3s-worker-3: [INFO]  Creating /usr/local/bin/kubectl symlink to k3s
v-k3s-worker-3: [INFO]  Creating /usr/local/bin/crictl symlink to k3s
v-k3s-worker-3: [INFO]  Creating /usr/local/bin/ctr symlink to k3s
v-k3s-worker-3: [INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
v-k3s-worker-3: [INFO]  Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
v-k3s-worker-3: [INFO]  env: Creating environment file /etc/systemd/system/k3s-agent.service.env
v-k3s-worker-3: [INFO]  systemd: Creating service file /etc/systemd/system/k3s-agent.service
v-k3s-worker-3: [INFO]  systemd: Enabling k3s-agent unit
v-k3s-worker-3: Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.
v-k3s-worker-3: [INFO]  systemd: Starting k3s-agent

这节点状态所有 Vagrantboxes 都启动并运行并且所有安装脚本完成后：

$ vagrant ssh v-k3s-master
$ sudo kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get nodes
NAME           STATUS   ROLES                  AGE   VERSION
v-k3s-master   Ready    control-plane,master   59m   v1.21.3+k3s1

当我通过浏览器从主机访问 https://localhost:6443 时，由于 SSL/HTTPS 设置不可信 ( Error code: SEC_ERROR_UNKNOWN_ISSUER)，我收到警告。响应来自主节点（唯一的可能性，因为没有连接从属代理）。据我所知，https://localhost:6443 是从属代理加入集群的首选地址。

由于我可以从从属虚拟机远程登录到主虚拟机，网络通信应该可以工作，但 curl 显示错误。此外，master-vm 只能通过 IP 访问，而不能通过其主机名访问：

vagrant@v-k3s-worker-3:~$ telnet 192.168.30.10 6443
Trying 192.168.30.10...
Connected to 192.168.30.10.
Escape character is '^]'.
^CConnection closed by foreign host.

vagrant@v-k3s-worker-3:~$ telnet v-k3s-master 6443
telnet: could not resolve v-k3s-master/6443: Temporary failure in name resolution

vagrant@v-k3s-worker-3:~$ curl https://192.168.30.10:6443
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
vagrant@v-k3s-worker-3:~$

我的完整Vagrant文​​件：

# -*- mode: ruby -*-
# vi: set ft=ruby :

IMAGE_NAME = "ubuntu/focal64" # focal = 20.04. LTS | bento/ubuntu-16.04
N = 3 # number of worker nodes

Vagrant.configure("2") do |config|
    config.ssh.insert_key = false

    # common for all vagrant boxes
    config.vm.provider "virtualbox" do |v|
        v.memory = 2048
        v.cpus = 2
        v.customize ["modifyvm", :id, "--groups", "/v-kube-cluster"]
    end

    # master node
    config.vm.define "v-k3s-master" do |master|
        master.vm.box = IMAGE_NAME
        master.vm.network "private_network", ip: "192.168.30.10"
        master.vm.hostname = "v-k3s-master"

        master.vm.provider "virtualbox" do |v|
            v.name = "v-k3s-master"
        end

        master.vm.network "forwarded_port", guest: 6443, host: 6443
        master.vm.network "forwarded_port", guest: 8001, host: 8001

        master.vm.provision "shell", path: "../common/provision/bash-setup.sh"
        master.vm.provision "shell", path: "../common/provision/install.sh"
        master.vm.provision "shell", path: "provision/master-node.sh"
    end

    # worker nodes
    (1..N).each do |i|
        config.vm.define "v-k3s-worker-#{i}" do |worker|
            worker.vm.box = IMAGE_NAME
            worker.vm.network "private_network", ip: "192.168.30.#{i + 10}"
            worker.vm.hostname = "v-k3s-worker-#{i}"

            worker.vm.provider "virtualbox" do |v|
                v.name = "v-k3s-worker-#{i}"
            end

            worker.vm.provision "shell", path: "../common/provision/bash-setup.sh"
            worker.vm.provision "shell", path: "../common/provision/install.sh"
            worker.vm.provision "shell", path: "provision/worker-node.sh"
        end
    end
end

我不知道如何解决这个问题。我怀疑 SSL 设置是问题所在，但我不知道如何解决这个问题。

或者这可能是一个问题，即每个虚拟机只能通过 IP 相互连接，而不能通过机器名（= 主机名）相互连接？那么可能存在 DNS 问题？

要查看所有脚本等，请参阅https://gitlab.com/sommerfeld.sebastian/v-kube-cluster/-/tree/feat/k3s/src/main/k3s... 这是我的项目的 repo（URL 指向相关目录）。

答案1

设置 --tls-san 和 --node-external-ip 参数即可解决问题。这样，k3s master 就可以监听其真实 IP，并通过将其 IP 放入证书中来接受请求。

主节点设置

masterIP="192.168.30.10"
k3sTokenFile="/var/lib/rancher/k3s/server/node-token"
flags="--tls-san $masterIP --node-external-ip $masterIP"

echo "[INFO] Install k3s on master-node"
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="$flags" K3S_NODE_NAME="$HOSTNAME" K3S_KUBECONFIG_MODE="644" sh -

echo "[INFO] Expose K3S_TOKEN from master-node for worker-node setup"
cp "$k3sTokenFile" /vagrant/resources/.generated/k3s.token

相关内容