☰

做一个平行循环

tag-icon • tag-icon linux shell parallelism
做一个平行循环

如何使这个循环并行：

for a in $(seq 1 3)
do
  for b in 0.1 0.2 
  do
    echo process with a=$a and b=$b &
  done
done

是并行还是不并行？事实上，我想echo process with a=$a and b=$b对每个值的组合并行运行指令a，b 这是运行上述 shell 的结果：

process with a=1 and b=0.1
process with a=2 and b=0.2
process with a=2 and b=0.1
process with a=3 and b=0.2
process with a=3 and b=0.1
process with a=1 and b=0.2

谢谢。

答案1

使用 GNU Parallel 时，它看起来像这样：

parallel echo process with a={1} and b={2} ::: 1 2 3 ::: 0.1 0.2 
seq 1 3 | parallel echo process with a={1} and b={2} :::: - ::: 0.1 
parallel echo process with a={1} and b={2} :::: <(seq 1 3) ::: 0.1 0.2

我认为echo这只是一个例子，因为并行化echo几乎不值得。

答案2

如果您在内循环中运行的内容花费了任意时间（echo即很快运行），然后，当循环启动所有异步进程时，它们将运行同时。

循环本身不是“并行的”。

答案3

这应该是一个评论，但评论太少了。

确保您的解决方案适用于任意大小的输出，并且不会将一项作业的输出与另一项作业的输出混合。这是一个小例子来测试：

#!/bin/bash                                                                               

# stress test of parallelizing programs                                                   

longline() {
    # print one big line with 'a's followed by 'b's followed by 'c's                      
    perl -e 'print "@ARGV ", map { "$_"x10000000 } (a..c)' "$@"
    echo
}


echo "Run testprogram in parallel"
for a in $(seq 1 3)
do
    for b in 0.1 0.2
    do
        longline $a $b &
    done
done |
# use 'tr' to compress many 'a's into a single 'a'                                        
# if the output is not 'abc' on every line, lines have been mixed                         
  tr -cs '@'

echo "Run testprogram in serial"
for a in $(seq 1 3)
do
    for b in 0.1 0.2
    do
        longline $a $b
    done
done | tr -cs '@'


echo "Compare with GNU Parallel"
export -f longline
parallel -kj0 longline :::: <(seq 1 3) ::: 0.1 0.2 | tr -cs '@'

相关内容