☰

运行异步任务并在 bash 中检索其退出代码和输出

tag-icon • tag-icon linux bash shell-script command-line process-substitution
运行异步任务并在 bash 中检索其退出代码和输出

我必须异步运行一堆 bash 命令，一旦命令完成，我需要根据其退出代码和输出执行操作。请注意，我无法预测这些任务在我的实际用例中将运行多长时间。

为了解决这个问题，我最终采用了以下算法：

For each task to be run:
    Run the task asynchronously;
    Append the task to the list of running tasks.
End For.

While there still are tasks in the list of running tasks:
    For each task in the list of running tasks:
        If the task has ended:
            Retrieve the task's exit code and output;
            Remove the task from the list of running tasks.
        End If.
    End For
End While.

这给了我以下 bash 脚本：

  1 #!/bin/bash
  2 
  3 # bg.sh
  4 
  5 # Executing commands asynchronously, retrieving their exit codes and outputs upon completion.
  6 
  7 asynch_cmds=
  8 
  9 echo -e "Asynchronous commands:\nPID    FD"
 10 
 11 for i in {1..10}; do
 12         exec {fd}< <(sleep $(( i * 2 )) && echo $RANDOM && exit $i) # Dummy asynchronous task, standard output's stream is redirected to the current shell
 13         asynch_cmds+="$!:$fd " # Append the task's PID and FD to the list of running tasks
 14         
 15         echo "$!        $fd"
 16 done    
 17 
 18 echo -e "\nExit codes and outputs:\nPID       FD      EXIT    OUTPUT"
 19 
 20 while [[ ${#asynch_cmds} -gt 0 ]]; do # While the list of running tasks isn't empty
 21         
 22         for asynch_cmd in $asynch_cmds; do # For each to in thhe list
 23                 
 24                 pid=${asynch_cmd%:*} # Task's PID
 25                 fd=${asynch_cmd#*:} # Task's FD
 26                 
 27                 if ! kill -0 $pid 2>/dev/null; then # If the task ended
 28                         
 29                         wait $pid # Retrieving the task's exit code
 30                         echo -n "$pid   $fd     $?      "
 31                         
 32                         echo "$(cat <&$fd)" # Retrieving the task's output
 33                         
 34                         asynch_cmds=${asynch_cmds/$asynch_cmd /} # Removing the task from the list
 35                 fi
 36         done
 37 done

输出告诉我wait尝试检索每个任务的退出代码失败，除了最后一个运行的:

Asynchronous commands:
PID     FD
4348    10
4349    11
4351    12
4353    13
4355    14
4357    15
4359    16
4361    17
4363    18
4365    19

Exit codes and outputs:
PID     FD  EXIT OUTPUT
./bg.sh: line 29: wait: pid 4348 is not a child of this shell
4348    10  127  16010
./bg.sh: line 29: wait: pid 4349 is not a child of this shell
4349    11  127  8341
./bg.sh: line 29: wait: pid 4351 is not a child of this shell
4351    12  127  13814
./bg.sh: line 29: wait: pid 4353 is not a child of this shell
4353    13  127  3775
./bg.sh: line 29: wait: pid 4355 is not a child of this shell
4355    14  127  2309
./bg.sh: line 29: wait: pid 4357 is not a child of this shell
4357    15  127  32203
./bg.sh: line 29: wait: pid 4359 is not a child of this shell
4359    16  127  5907
./bg.sh: line 29: wait: pid 4361 is not a child of this shell
4361    17  127  31849
./bg.sh: line 29: wait: pid 4363 is not a child of this shell
4363    18  127  28920
4365    19  10   28810

命令的输出被完美地检索到，但我不明白这个is not a child of this shell错误来自哪里。我一定做错了什么，因为wait能够获取要异步运行的最后一个命令的退出代码。

有谁知道这个错误来自哪里？我对这个问题的解决方案是否有缺陷，或者我是否误解了 bash 的行为？我很难理解 的行为wait。

PS：我在 Super User 上发布了这个问题，但转念一想，它可能更适合 Unix & Linux Stack Exchange。

答案1

这是一个错误/限制； bash 只允许等待最后一个进程替换，无论您是否将 的值保存$!到另一个变量中。

更简单的测试用例：

$ cat script
exec 7< <(sleep .2); pid7=$!
exec 8< <(sleep .2); pid8=$!
echo $pid7 $pid8
echo $(pgrep -P $$)
wait $pid7
wait $pid8

$ bash script
6030 6031
6030 6031
/tmp/sho: line 9: wait: pid 6030 is not a child of this shell

尽管pgrep -P实际上发现它是 shell 的子级，并且strace表明它bash实际上正在收获它。

但无论如何，$!也设置为最后一个进程替换的 PID 是一个未记录的功能（iirc 没有在旧版本中使用该功能），并且受到一些限制陷阱。

发生这种情况是因为 bash 仅跟踪last_procsub_child变量中的最后一个进程替换。这是wait寻找 pid 的地方：

-- jobs.c --
/* Return the pipeline that PID belongs to.  Note that the pipeline
   doesn't have to belong to a job.  Must be called with SIGCHLD blocked.
   If JOBP is non-null, return the index of the job containing PID.  */
static PROCESS *
find_pipeline (pid, alive_only, jobp)
     pid_t pid;
     int alive_only;
     int *jobp;         /* index into jobs list or NO_JOB */
{
     ...
  /* Now look in the last process substitution pipeline, since that sets $! */
  if (last_procsub_child)
    {

但当创建新的 proc subst 时，它将被丢弃：

-- subst.c --
static char *
process_substitute (string, open_for_read_in_child)
     char *string;
     int open_for_read_in_child;
{
   ...
      if (last_procsub_child)
        discard_last_procsub_child ();

答案2

这就是我想出来的。

首先，一个虚拟run脚本，在您的情况下将是完全不同的：

#!/bin/bash

sleep $1;
exit $2

接下来，一个bg脚本将run作业放入后台，并进行适当的重定向：

#!/bin/bash

echo $$

( ( touch $$.running; "$@" > $$.out 2>$$.err ; echo $? > $$.exitcode ) & )

最后，一个driver控制整个事情的脚本。这是您将实际运行的脚本，当然不是其他两个。里面的评论应该有帮助，但我已经测试过它，它似乎工作得很好。

#!/bin/bash

# first run all commands via "bg"
./bg ./run 10 0
./bg ./run 5 5
./bg ./run 2 2
./bg ./run 0 0
# ... and so on

while :
do
    shopt -s nullglob
    for i in *.exitcode
    do
        j=$(basename $i .exitcode)
        # now process $j.out, $j.err, $j.exitcode however you want; most
        # importantly, *move* at least the exitcode file out of this directory
        echo $j had exit code of `cat $i`
        rm $j.*
    done

    shopt -u nullglob
    ls *.running >/dev/null 2>&1 || exit
    sleep 1
done

相关内容