☰

xargs 不会因错误而退出

tag-icon • tag-icon xargs exit-status
xargs 不会因错误而退出

我有一个 bash 脚本，我使用find来获取目录中的一堆文件，然后用它xargs在 chroot 环境中一次执行 1 个脚本中的所述文件。我的理解是，一旦 xargs 看到非零退出代码，它就会退出并停止处理，但是，由于某种原因，情况似乎并非如此。

我有的脚本：

#!/usr/bin/env bash

set -euo pipefail

script_dir=$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )

rootfs="$1"

exec_script() {
  script="$1"

  relative_script_path="$(realpath --relative-to="$script_dir" "$script")"

  echo -e "\e[1;94m=> executing script $script ($relative_script_path)\e[0m"

  sleep 5

  if ! "$rootfs/enter-chroot" sh -c "/$relative_script_path"; then
    echo -e "\e[1;31m=> script $script failed\e[0m"
    exit 1
  fi

  echo -e "\e[1;32m=> script $script ran successfully\e[0m"
}

export -f exec_script
export rootfs
export script_dir

find "$script_dir/build/scripts" -name '*.sh' -print0 | sort -z | xargs -r -0 -I% -n1 bash -c 'exec_script "$@"' _ %

当我运行它时，我得到以下输出：

./build/run.sh /tmp/test
=> executing script /tmp/builder/build/scripts/000-upgrade.sh (build/scripts/000-upgrade.sh)
environment: line 4: /tmp/test/enter-chroot: Not a directory
=> script /tmp/builder/build/scripts/000-upgrade.sh failed
=> executing script /tmp/builder/build/scripts/001-firmware.sh (build/scripts/001-firmware.sh)
environment: line 4: /tmp/test/enter-chroot: Not a directory
=> script /tmp/builder/build/scripts/001-firmware.sh failed

我哪里错了？如何确保 xargs 停止处理并以非零退出代码退出？

答案1

的文档xargs（参见man xargs实际上说明了有关退出的内容，

如果该命令的任何调用以状态 255 退出，xargs将立即停止，而不读取任何进一步的输入。错误消息发出于标准错误当这个情况发生时。

因此，一种可能的解决方案是更改exec_script为出错时返回退出状态 255。

在无法更改的情况下，另一种可能的解决方案exec_script是将普通内容转换xargs为 shell 循环：

find "$script_dir/build/scripts" -name '*.sh' -print0 |
    sort -z |
    while IFS= read -r -d '' item && exec_script _ "$item"; do :; done

exec_script如果返回任何非零退出值，循环将中断。

另一种解决方案，来自评论这可以说是最简单的外部修复，就是捕获脚本中的任何退出错误并将其替换为 255：

find "$script_dir/build/scripts" -name '*.sh' -print0 |
    sort -z |
    xargs -r -0 -I% -n1 bash -c 'exec_script "$@" || exit 255' _ %

答案2

xargs相当麻烦，因为它只有在以 255 退出时才会停止，那么，set -eo pipefail导出函数时 using 将完全无效，除非您使用子 shell 来封装set -eo pipefail失败并用 传播它255。

在此示例中，注释掉该errornow行以查看差异：

#!/bin/bash
set -x
set -eo pipefail

parallel_uploads="4"
s3_bucket_name="backup"

all_files=(
"/12.pbd"
"/13.pbd"
"/14.pbd"
"/15.pbd"
"/16.pbd"
"/17.pbd"
"/18.pbd"
"/19.pbd"
"/20.pbd"
"/21.pbd"
"/22.pbd"
"/23.pbd"
"/24.pbd"
"/25.pbd"
"/26.pbd"
"/27.pbd"
)


# Workaround for the posix shell bug they call it feature
# https://unix.stackexchange.com/questions/65532/why-does-set-e-not-work-inside-subshells-with-parenthesis-followed-by-an-or
function acually_upload_to_s3()
{
    set -x;
    set -eu -o pipefail;

    printf 'Doing some\n';
    sleeptime="$(( RANDOM % 50 + 1 ))"
    sleep "$sleeptime";

    erroring_some;
    printf 'Doing more some\n';
}

function upload_to_s3()
{
    set -x;
    set -eu -o pipefail;
    # https://superuser.com/questions/403263/how-to-pass-bash-script-arguments-to-a-subshell
    /bin/bash -c "acually_upload_to_s3 $(printf "${1+ %q}" "$@")" || exit 255
}

function upload_all()
{
    export s3_bucket_name;
    export -f upload_to_s3;
    export -f acually_upload_to_s3;

    # https://unix.stackexchange.com/questions/566834/xargs-does-not-quit-on-error
    # https://stackoverflow.com/questions/11003418/calling-shell-functions-with-xargs
    # https://stackoverflow.com/questions/6441509/how-to-write-a-process-pool-bash-shell
    # https://stackoverflow.com/questions/356100/how-to-wait-in-bash-for-several-subprocesses-to-finish-and-return-exit-code-0
    printf "'%s'\n" "${all_files[@]}" | xargs \
            --max-procs="$parallel_uploads" \
            --max-args=1 \
            --replace={} \
            /bin/bash -c 'time upload_to_s3 "$s3_bucket_name" "{}"';
}

time upload_all \
    && printf '%s Successfully uploaded all files\n' "$(date)" \
    || printf '%s Error: Could not upload some files\n'  "$(date)";

输出示例：

$ bash upload_to_s3_glacier_deep.sh
+ set -eo pipefail
+ parallel_uploads=4
+ s3_bucket_name=backup
+ all_files=("/12.pbd" "/13.pbd" "/14.pbd" "/15.pbd" "/16.pbd" "/17.pbd" "/18.pbd" "/19.pbd" "/20.pbd" "/21.pbd" "/22.pbd" "/23.pbd" "/24.pbd" "/25.pbd" "/26.pbd" "/27.pbd")
+ upload_all
+ export s3_bucket_name
+ export -f upload_to_s3
+ export -f acually_upload_to_s3
+ printf ''\''%s'\''\n' /12.pbd /13.pbd /14.pbd /15.pbd /16.pbd /17.pbd /18.pbd /19.pbd /20.pbd /21.pbd /22.pbd /23.pbd /24.pbd /25.pbd /26.pbd /27.pbd
+ xargs --max-procs=4 --max-args=1 '--replace={}' /bin/bash -c 'time upload_to_s3 "$s3_bucket_name" "{}"'
+ set -eu -o pipefail
++ printf ' %q' backup /12.pbd
+ /bin/bash -c 'acually_upload_to_s3  backup /12.pbd'
+ set -eu -o pipefail
++ printf ' %q' backup /13.pbd
+ /bin/bash -c 'acually_upload_to_s3  backup /13.pbd'
+ set -eu -o pipefail
++ printf ' %q' backup /14.pbd
+ /bin/bash -c 'acually_upload_to_s3  backup /14.pbd'
+ set -eu -o pipefail
+ printf 'Doing some\n'
Doing some
+ sleeptime=3
+ sleep 3
+ set -eu -o pipefail
++ printf ' %q' backup /15.pbd
+ /bin/bash -c 'acually_upload_to_s3  backup /15.pbd'
+ set -eu -o pipefail
+ printf 'Doing some\n'
Doing some
+ sleeptime=49
+ sleep 49
+ set -eu -o pipefail
+ printf 'Doing some\n'
Doing some
+ sleeptime=13
+ sleep 13
+ set -eu -o pipefail
+ printf 'Doing some\n'
Doing some
+ sleeptime=30
+ sleep 30
+ erroring_some
environment: line 5: erroring_some: command not found
+ exit 255

real    0m3.146s
user    0m0.045s
sys 0m0.123s
xargs: /bin/bash: exited with status 255; aborting
+ erroring_some
environment: line 5: erroring_some: command not found
+ exit 255

real    0m13.149s
user    0m0.015s
sys 0m0.123s
xargs: /bin/bash: exited with status 255; aborting

real    0m13.271s
user    0m0.075s
sys 0m0.337s
++ date
+ printf '%s Error: Could not upload some files\n' 'Fri, Nov 19, 2021 22:00:30'
Fri, Nov 19, 2021 22:00:30 Error: Could not upload some files

相关内容