☰

在复制到所有远程服务器之前验证所有文件

tag-icon • tag-icon linux ubuntu ansible ansible-playbook
在复制到所有远程服务器之前验证所有文件

我有一个 ansible 剧本，它由两个剧本组成，如下所示：

  • 在调用 Play2 之前，第一个 play 会验证我运行 ansible 的本地框上的所有内容。如果我在 Play1 中的验证失败，那么我根本不想启动 Play2。
  • 仅当 Play1 验证成功时，第二次播放才会启动，并且如果远程服务器上不存在目录，则此播放会在远程服务器上创建一个目录，然后将process.tar.gz文件复制到所有远程服务器上的特定目录中。

以下是我的剧本：

---
- name: Play 1
  hosts: 127.0.0.1
  tasks:
      - name: check whether we have all the necessary files
        shell: "ls files/ | wc -l"
        retries: 10
        delay: 10
        register: number_of_files
        until: "number_of_files.stdout == '10'"

      - name: Total number of files
        debug: msg="Total files {{ number_of_files.stdout }}"

      - name: check whether all the files are generated within a minute difference
        shell: "find files/ -mmin -1 -type f -print | wc -l"
        register: file_time
        failed_when: "file_time.stdout != '10'"

      - name: Total number of files
        debug: msg="Total files {{ file_time }}"

      - name: check whether success file was generated
        shell: "ls files/SUCCESS | wc -l"
        register: success_file
        failed_when: "success_file.stdout != '1'"

      - name: Success file
        debug: msg="{{ success_file }}"

      - name: compress all the files in tar.gz
        shell: "rm process.tar.gz; tar -czvf process.tar.gz -C files . --exclude='SUCCESS'"

- name: Play 2
  hosts: ALL_HOSTS
  serial: "{{ serial }}"
  tasks:
      - name: creates directory
        file: path=/data/taks/files/ state=directory

      - name: copy and untar process.tar.gz file
        unarchive: src=process.tar.gz dest=/data/taks/files/

      - name: sleep for few seconds
        pause: seconds=20

我想看看是否有更好的方法来编写上述 ansible？由于我最近才开始使用 ansible，所以不确定我是否遵循了所有最佳实践。我正在运行 ansible 版本ansible 2.4.3.0。此外，在执行此任务时我总是收到来自 ansible 的警告：

- name: compress all the files in tar.gz
  shell: "rm process.tar.gz; tar -czvf process.tar.gz -C files . --exclude='SUCCESS'"

以下是警告：

[WARNING]: Consider using file module with state=absent rather than running rm
[WARNING]: Consider using unarchive module rather than running tar

更新：

下面是我的 ansible playbook 所在的路径，我只能从这个目录执行我的PLAYBOOK1。files目录包含SUCCESS我需要压缩成 tar.gz 的所有文件。

jenkins@machineA:~/jobs/processdata/workspace$ ls -lrth
total 145M
drwxr-xr-x 2 jenkins jenkins 4.0K Feb 19 17:36 files
-rw-r--r-- 1 jenkins root    1.6K Feb 19 19:32 PLAYBOOK1.yml
-rw-r--r-- 1 jenkins root    1.6K Feb 19 19:32 PLAYBOOK2.yml

PLAYBOOK1这是我执行您的更改后收到的错误：

TASK [Check all files] **********************************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "examined": 0, "failed_when_result": true, "files": [], "matched": 0, "msg": "/files was skipped as it does not seem to be a valid directory or it cannot be accessed\n"}

答案1

请参阅我的评论，标记为#

剧本1

- hosts: YOURHOST1
  remote_user: "YOURUSER"
  become: True

  tasks:

  #Instead using shell, use find module... It will check if there are 10 files created before 1 minute in folder, otherwise playbook fails
  - name: Check all files
    find: paths=/var/lib/jenkins/jobs/processdata/workspace/files
          file_type=file
          age=-1m
          age_stamp=mtime
    register: files
    failed_when: files.matched < 10

  - name: Number of files
    debug: msg="Total files {{ files.matched }}"

  #Check for SUCCESS file, I didn't add an action to delete SUCCESS file... Hope you have any other script to do it
  - name: Check Success File
    find: paths=/var/lib/jenkins/jobs/processdata/workspace/files
          file_type=file
          patterns=SUCCESS
    register: success_file
    failed_when: success_file.matched < 1

  - name: Success file
    debug: msg='Success file is {{ success_file.files.0.path }}'

  - name: Remove previous tarFile
    file: path=/tmp/process.tar.gz
          state=absent

  #I've found a way to make it work
  - name: compress all the files in tar.gz
    archive: path="{{ files_to_archive }}"
             dest=/tmp/test.tar.gz
             format=gz
    vars:
        files_to_archive: "{{ files.files | map(attribute='path') | list }}"

- include: PLAYBOOK2.yml

剧本2如果上述某个操作失败，Ansible 将停止并且不会运行任何其他操作。

- hosts: YOURHOST2
  remote_user: "YOURUSER"
  become: True

  tasks:

  - name: creates directory
    file: path=/data/taks/files/
          state=directory

  - name: Copy tar file
    copy: src=/tmp/test.tar.gz
          dest=/tmp/process.tar.gz
          owner=root
          group=root
          mode=0744

  - name: copy and untar process.tar.gz file
    unarchive: src=/tmp/process.tar.gz
               dest=/data/taks/files/

  - name: Remove SUCCESS File
    file: path=/data/taks/files/SUCCESS
          state=absent

#I don't know why you'll need this:
  - name: sleep for few seconds
    pause: seconds=20

请考虑 ansible 的一个特性是幂等性，你应该编写脚本来运行并从之前的操作继续（如果它们在之前的启动中失败）。你的脚本和我的脚本都不是幂等的，因为你需要找到一分钟前更新的文件，因此在下次启动时（如果出现故障），你的脚本会丢弃一分钟前的文件。

SUCCESS 文件也会发生同样的情况，如果您有一个平台/脚本正在继续删除它，则 Ansible 脚本在运行时可能会失败。

相关内容