![Bash 等待作业并限制作业数量](https://linux22.com/image/24938/Bash%20%E7%AD%89%E5%BE%85%E4%BD%9C%E4%B8%9A%E5%B9%B6%E9%99%90%E5%88%B6%E4%BD%9C%E4%B8%9A%E6%95%B0%E9%87%8F.png)
可能的重复:
并行执行四个任务……我该怎么做?
假设一个循环调用一个命令
grep -v '#' < files.m3u | sed 's/\\\\/\/\//g' | sed 's/\\/\//g' | while read line
do
filename=$(basename "$line")
avconv -i "$line" "${filename%.*}.wav"
done
在 avconv 之后放置 & 将为每个文件持续生成 avconv。现在我想做两件事:
- 我想将生成的进程数限制为 4
- 循环完成后,我想等待最后一个准备好
答案1
您可以记住每个新子进程的 PID($!
启动后检查)。定期检查仍有多少子级存在(例如通过kill -0
),如果数量下降,则生成一个新的,等等。最后,只需wait
。
这是我出于同样原因编写的脚本:
#! /bin/bash
## Tries to run commands in parallel. Commands are read from STDIN one
## per line, or from a given file specified by -f.
## Author: E. Choroba
file='-'
proc_num=$(grep -c ^processor'\b' /proc/cpuinfo)
prefix=$HOSTNAME-$USER-$$
sleep=10
children=()
names=()
if [[ $1 =~ ^--?h(elp)?$ ]] ; then
cat <<-HELP
Usage: ${0##*/} [-f file] [-n max-processes] [-p tmp-prefix] -s [sleep]
Defaults:
STDIN for file
$proc_num for max-processes (number of processors)
$prefix for tmp-prefix
$sleep for sleep interval
HELP
exit
fi
function debug () {
if ((DEBUG)) ; then
echo "$@" >&2
fi
}
function child_count () {
debug Entering child_count "${children[@]}"
child_count=0
new_children=()
for child in "${children[@]}" ; do
debug Trying $child
if kill -0 $child 2>/dev/null ; then
debug ... exists
let child_count++
new_children+=($child)
fi
done
children=("${new_children[@]}")
echo $child_count
debug Leaving child_count "${children[@]}"
}
while getopts 'f:n:p:s:' arg ; do
case $arg in
f ) file=$OPTARG ;;
n ) proc_num=$((OPTARG)) ;;
p ) prefix=$OPTARG;;
s ) sleep=$OPTARG;;
* ) echo "Warning: unknown option $arg" >&2 ;;
esac
done
i=0
while read -r line ; do
debug Reading $line
name=$prefix.$i
let i++
names+=($name)
while ((`child_count`>=proc_num)) ; do
sleep $sleep
debug Sleeping
done
eval $line 2>$name.e >$name.o &
children+=($!)
debug Running "${children[@]}"
done < <(cat $file)
debug Loop ended
wait
cat "${names[@]/%/.o}"
cat "${names[@]/%/.e}" >&2
rm "${names[@]/%/.o}" "${names[@]/%/.e}"
答案2
来自链接问题,根据您的变化量身定制:
sed -n -e '/#/!s,\\,/,g' files.m3u | xargs -d '\n' -I {} -P 4 \
sh -c 'line=$1; file=${line##*/}; avconv -i "$line" "${file%.*}.wav"' avconv_sh {}
同样,GNUxargs
或某些版本支持-d
和-P
是必需的。还要注意输入文件中行首和行尾的多余空格 - 如果它们存在,此代码片段将保留它们,这可能会导致问题。
答案3
我是这样解决的。谢谢你的美元!尖端
#!/bin/bash
children[0]=0
children[1]=0
children[2]=0
children[3]=0
i=0
grep -v '#' < files.m3u | sed 's/\\\\/\/\//g' | sed 's/\\/\//g' | while read line
do
filename=$(basename "$line")
let k="$i%4"
wait ${children[k]}
avconv -i "$line" "${filename%.*}.wav" &
children[k]=$!
let i++
done
wait ${children[0]}
wait ${children[1]}
wait ${children[2]}
wait ${children[3]}