☰

为 Prometheus 创建配置文件

tag-icon • tag-icon ansible
为 Prometheus 创建配置文件

库存文件（inventories/test/environments/tmp6.yml）：

test:
  children:
    environments:
      children:
        tmp6:

tmp6:
  vars:
    ansible_user: superuser
    ansible_password: superuser
    ansible_become_password: "{{ ansible_password }}"
  children:
    infra:
      hosts:
        host_5:
          ansible_host: 192.168.1.72
        host_6:
          ansible_host: 192.168.1.83
    tools:
      hosts:
        host_7:
          ansible_host: 192.168.1.239
        host_8:
          ansible_host: 192.168.1.46

一般的playbook.yml：

- hosts: environments
  roles:
    - role: host-configuration
    - role: host-users
    - role: node-exporter
  post_tasks:
    - ansible.builtin.copy:
        content: |
          - target: 
              - {{ ansible_host }}:9100
            labels:
              environment_name: {{ inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '') }}
              environment_type: {{ inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}
        dest: "./{{ inventory_file | ansible.builtin.basename }}"
      delegate_to: localhost

在节点上成功安装node_exporter后，需要为环境创建一个通用的配置文件：

- target:
    - 192.168.1.72:9100
    - 192.168.1.83:9100
    - 192.168.1.239:9100
    - 192.168.1.46:9100
  labels:
    environment_name: tmp6
    environment_type: test

主要问题在于制定目标清单。

如果创建这样的模板：

- target: 
    - {{ ansible_host }}:9100
  labels:
    environment_name: {{ inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '') }}
    environment_type: {{ inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}

变量的值ansible_host总是在变化。因此这不是一个解决方案。

如何在剧本中创建动态主机列表？

PS：Playbook可以应用于多种环境：

ansible-playbook playbook.yml -i inventories/test -l tmp1,tmp3,tmp6

常规任务：

  1. 在“环境”组的节点上安装 node_exporter
  2. 在本地主机上创建通用配置文件
  3. 将配置文件复制到主机“prometheus”

答案1

如果使用多个环境，则必须创建多个文件

Ansible 对您的“环境”一无所知。如果您想在剧本中使用该信息，则需要将其添加到您的库存中。例如，我们可以添加environment_type和environment_name变量，如下所示：

test:
  vars:
    environment_type: test
  children:
    environments:
      children:
        tmp5:
        tmp6:

tmp5:
  vars:
    environment_name: tmp5
  children:
    tmp5_infra:
      hosts:
        host_1:
          ansible_host: 192.168.1.1
        host_2:
          ansible_host: 192.168.1.2
    tmp5_tools:
      hosts:
        host_3:
          ansible_host: 192.168.1.3
        host_4:
          ansible_host: 192.168.1.4

tmp6:
  vars:
    environment_name: tmp6
  children:
    tmp6_infra:
      hosts:
        host_5:
          ansible_host: 192.168.1.5
        host_6:
          ansible_host: 192.168.1.6
    tmp6_tools:
      hosts:
        host_7:
          ansible_host: 192.168.1.7
        host_8:
          ansible_host: 192.168.1.8

鉴于上述库存文件，我们可以编写如下剧本：

- hosts: environments
  gather_facts: false
  tasks:

    # This task groups hosts by environment_name
    - delegate_to: localhost
      run_once: true
      loop: "{{ ansible_play_hosts_all }}"
      set_fact:
        env_to_host: >
          {{ env_to_host|combine({
            hostvars[item].environment_name: env_to_host.get(hostvars[item].environment_name, []) + [item]
          })}}
      vars:
        env_to_host: {}

    - delegate_to: localhost
      run_once: true
      loop: "{{ env_to_host.keys() }}"
      copy:
        content: |
          - target: 
          {% for host in env_to_host[item] %}
              - {{ hostvars[host].ansible_host }}:9100
          {% endfor %}
            labels:
              environment_name: {{ item }}
              environment_type: {{ hostvars[env_to_host[item]|first].environment_type }}
        dest: "./config-{{item}}.yaml"

如果我们运行：

ansible-playbook playbook.yaml

我们得到：

$ ls config*
config-tmp5.yaml  config-tmp6.yaml
$ cat config-tmp5.yaml
- target:
    - 192.168.1.1:9100
    - 192.168.1.2:9100
    - 192.168.1.3:9100
    - 192.168.1.4:9100
  labels:
    environment_name: tmp5
    environment_type: test
$ cat config-tmp6.yaml
- target:
    - 192.168.1.5:9100
    - 192.168.1.6:9100
    - 192.168.1.7:9100
    - 192.168.1.8:9100
  labels:
    environment_name: tmp6
    environment_type: test

如果我们受环境限制，就像这样：

ansible-playbook playbook.yaml -l tmp5

我们得到：

$ ls config*
config-tmp5.yaml
$ cat config-tmp5.yaml
- target:
    - 192.168.1.1:9100
    - 192.168.1.2:9100
    - 192.168.1.3:9100
    - 192.168.1.4:9100
  labels:
    environment_name: tmp5
    environment_type: test

最后，如果我们限制到单个主机，如下所示：

ansible-playbook playbook.yaml -l host_1,host_5

我们得到：

$ ls config*
config-tmp5.yaml  config-tmp6.yaml
$ cat config-tmp5.yaml
- target:
    - 192.168.1.1:9100
  labels:
    environment_name: tmp5
    environment_type: test
$ cat config-tmp6.yaml
- target:
    - 192.168.1.5:9100
  labels:
    environment_name: tmp6
    environment_type: test

答案2

该问题可以通过中间临时文件解决。

  1. 创建临时文件：
- ansible.builtin.file:
    path: "./.configs_prometheus/{{
           inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}_--_{{
           ansible_host }}_--_{{ inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '') }}"
    state: touch
  1. 搜索临时文件
- ansible.builtin.find:
    paths: "./.configs_prometheus/"
    file_type: file
    recurse: yes
  register: environment_temp_files_list
  1. 生成数据列表：
- ansible.builtin.set_fact:
    environment_data: "{{ environment_temp_files_list.files | map(attribute='path') | map('ansible.builtin.basename') |
                          map('ansible.builtin.split','_--_') | map('ansible.builtin.zip', ['type', 'host', 'name']) |
                          map('map', 'reverse') | map('community.general.dict') | groupby('name') }}"

- ansible.builtin.set_fact: environments_list="{{ environment_data | map('first') }}"

- ansible.builtin.set_fact: hosts_list="{{ environment_data | map('last') | map('map', attribute='host') }}"

- ansible.builtin.set_fact: types_list="{{ environment_data | map('last') | map('map', attribute='type') | map('ansible.builtin.unique') }}"
  1. 模板生成：
- ansible.builtin.copy:
    content: |
      - targets:
        {%  for result in item.1.0 %}
        - {{ result }}:9100
        {% endfor %}
      labels:
          environment_name: {{ item.0 }}
          environment_type: {{ item.1.1.0 }}
    dest: "./.configs_prometheus/{{ item.1.1.0 }}_--_{{ item.0 }}.yml"
  loop: "{{ environments_list | zip( hosts_list | zip(types_list) ) }}"
  loop_control:
    label: "{{ item.0 }}"

最终剧本： 

- hosts: localhost
  tasks:
    - name: Creating a directory
      ansible.builtin.file:
        path: "./.configs_prometheus"
        state: "{{ item }}"
      delegate_to: localhost
      loop:
        - absent
        - directory

- hosts: environments
  roles:
    - role: host-configuration
    - role: host-users
    - role: node-exporter
  post_tasks:
    - name: Creating temporary files
      ansible.builtin.file:
        path: "./.configs_prometheus/{{
               inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}_--_{{
               ansible_host }}_--_{{ inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '') }}"
        state: touch
      delegate_to: localhost

- hosts: localhost
  tasks:
    - name: Search for temporary files
      ansible.builtin.find:
        paths: "./.configs_prometheus/"
        file_type: file
        recurse: yes
      register: environment_temp_files_list

    - name: Generating data lists
      ansible.builtin.set_fact:
        environment_data: "{{ environment_temp_files_list.files | map(attribute='path') | map('ansible.builtin.basename') |
                              map('ansible.builtin.split','_--_') | map('ansible.builtin.zip', ['type', 'host', 'name']) |
                              map('map', 'reverse') | map('community.general.dict') | groupby('name') }}"
    - ansible.builtin.set_fact: environments_list="{{ environment_data | map('first') }}"
    - ansible.builtin.set_fact: hosts_list="{{ environment_data | map('last') | map('map', attribute='host') }}"
    - ansible.builtin.set_fact: types_list="{{ environment_data | map('last') | map('map', attribute='type') | map('ansible.builtin.unique') }}"

    - name: Template generation
      ansible.builtin.copy:
        content: |
          - targets:
            {%  for result in item.1.0 %}
            - {{ result }}:9100
            {% endfor %}
          labels:
              environment_name: {{ item.0 }}
              environment_type: {{ item.1.1.0 }}
        dest: "./.configs_prometheus/{{ item.1.1.0 }}_--_{{ item.0 }}.yml"
      loop: "{{ environments_list | zip( hosts_list | zip(types_list) ) }}"
      loop_control:
        label: "{{ item.0 }}"

    - name: Deleting temporary files
      ansible.builtin.file:
        path: "{{ item }}"
        state: absent
      loop: "{{ environment_temp_files_list.files | map(attribute='path') }}"
      loop_control:
        label: "{{ item.split('/').1 }}"

更新

无需中间临时文件即可解决问题。

  1. 创建环境数据集：
- ansible.builtin.set_fact:
    environment_dataset: |
      {% for play_host in ansible_play_hosts_all %}
        - {{ hostvars[play_host].inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}_--_{{
             hostvars[play_host].ansible_host }}_--_{{
             hostvars[play_host].inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '')
          }}
      {% endfor %}
  run_once: true
  delegate_to: "{{ item }}"
  delegate_facts: true
  loop: "{{ groups['prometheus'] | default(['localhost']) }}"
  1. 生成数据列表：
- ansible.builtin.set_fact:
    environment_data: "{{ environment_dataset | from_yaml |
                          map('ansible.builtin.split','_--_') | map('ansible.builtin.zip', ['type', 'host', 'name']) |
                          map('map', 'reverse') | map('community.general.dict') | groupby('name') }}"

- ansible.builtin.set_fact: environments_list="{{ environment_data | map('first') }}"

- ansible.builtin.set_fact: hosts_list="{{ environment_data | map('last') | map('map', attribute='host') }}"

- ansible.builtin.set_fact: types_list="{{ environment_data | map('last') | map('map', attribute='type') | map('ansible.builtin.unique') }}"
  1. 模板生成：
- ansible.builtin.copy:
    content: |
      - targets:
        {%  for result in item.1.0 %}
        - {{ result }}:9100
        {% endfor %}
      labels:
          environment_name: {{ item.0 }}
          environment_type: {{ item.1.1.0 }}
    dest: "/etc/prometheus/file_sd_configs/{{ item.1.1.0 }}_--_{{ item.0 }}.yml"
  loop: "{{ environments_list | zip( hosts_list | zip(types_list) ) }}"
  loop_control:
    label: "{{ item.0 }}"

最终剧本： 

- hosts: environments
  roles:
    - role: host-configuration
    - role: host-users
    - role: node-exporter
  post_tasks:
    - name: Creating temporary files
      ansible.builtin.set_fact:
        environment_dataset: |
          {% for play_host in ansible_play_hosts_all %}
            - {{ hostvars[play_host].inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}_--_{{
                 hostvars[play_host].ansible_host }}_--_{{
                 hostvars[play_host].inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '')
              }}
          {% endfor %}
      run_once: true
      delegate_to: "{{ item }}"
      delegate_facts: true
      loop: "{{ groups['prometheus'] | default(['localhost']) }}"

- hosts: prometheus
  tasks:
    - name: Generating data lists
      ansible.builtin.set_fact:
        environment_data: "{{ environment_dataset | from_yaml |
                              map('ansible.builtin.split','_--_') | map('ansible.builtin.zip', ['type', 'host', 'name']) |
                              map('map', 'reverse') | map('community.general.dict') | groupby('name') }}"
    - ansible.builtin.set_fact: environments_list="{{ environment_data | map('first') }}"
    - ansible.builtin.set_fact: hosts_list="{{ environment_data | map('last') | map('map', attribute='host') }}"
    - ansible.builtin.set_fact: types_list="{{ environment_data | map('last') | map('map', attribute='type') | map('ansible.builtin.unique') }}"

    - name: Template generation
      ansible.builtin.copy:
        content: |
          - targets:
            {%  for result in item.1.0 %}
            - {{ result }}:9100
            {% endfor %}
          labels:
              environment_name: {{ item.0 }}
              environment_type: {{ item.1.1.0 }}
        dest: "/etc/prometheus/file_sd_configs/{{ item.1.1.0 }}_--_{{ item.0 }}.yml"
      become: yes
      loop: "{{ environments_list | zip( hosts_list | zip(types_list) ) }}"
      loop_control:
        label: "{{ item.0 }}"

相关内容