重用变量

Question 1

您可能需要这样的东西来解析您的日志：

#!/bin/awk -f
{
    if (/was successful/) {
        bn = $2;
        gsub(/'/, "", bn);
        succ[bn] = 1;
    }
    if (/Processing batch/) {
        bn = $7;
        gsub(/'/, "", bn);
        if (bn in succ) {
            succ[bn,",s"] = $2;
            getline
            succ[bn,",e"] = $2;
        } else
            fail[bn] = 1;
    }
}
END {
    for (val in succ)
        if (val !~ /,/)
            print val " " succ[val,",s"] " " succ[val,",e"];
    for (val in fail)
        print val " encountered an error";
}

你原来的剧本轻微地减少到仅1awk在有关输入数据结构的一些严格的¹假设下调用）。

#!/bin/ksh

fname=$1

for batchname in $(grep -i "Processing batch" $fname | cut -d "'" -f2); do
    Batch_state=`grep -c -i "Batch '$batchname' was successful" $fname`
    if [[ "$Batch_state"  -ge 1 ]]; then
        awk -v b=$batchname '
{
    if ($0 ~ "Processing.*" b) {
        s = $2;
        getline;
        e = $2;
        print b " " s " " e;
        exit
    }
}' $fname
    else
        echo $batchname encountered an error
    fi
done

grep -A使用(的较短版本-A是 GNU 扩展，它选择匹配行和后面的行数（默认为 1）：

#!/bin/ksh

fname=$1

for batchname in $(grep -i "Processing batch" $fname | cut -d "'" -f2); do
    Batch_state=`grep -c -i "Batch '$batchname' was successful" $fname`
    printf "%s" $batchname
    if [[ "$Batch_state"  -ge 1 ]]; then
        grep -A1 "Processing batch '$batchname'" $fname | cut -f2 -d" " | fmt
    else
        printf "encountered an error\n"
    fi
done

但这些仍在处理文件3次。请参阅 ChuckCottrill 的回答了解如何完成。

¹它似乎批次不是并行处理的，您想要的实际上是包含该行Processing batch及其之后的行。

Answer

您可能需要这样的东西来解析您的日志：

#!/bin/awk -f
{
    if (/was successful/) {
        bn = $2;
        gsub(/'/, "", bn);
        succ[bn] = 1;
    }
    if (/Processing batch/) {
        bn = $7;
        gsub(/'/, "", bn);
        if (bn in succ) {
            succ[bn,",s"] = $2;
            getline
            succ[bn,",e"] = $2;
        } else
            fail[bn] = 1;
    }
}
END {
    for (val in succ)
        if (val !~ /,/)
            print val " " succ[val,",s"] " " succ[val,",e"];
    for (val in fail)
        print val " encountered an error";
}

你原来的剧本轻微地减少到仅1awk在有关输入数据结构的一些严格的¹假设下调用）。

#!/bin/ksh

fname=$1

for batchname in $(grep -i "Processing batch" $fname | cut -d "'" -f2); do
    Batch_state=`grep -c -i "Batch '$batchname' was successful" $fname`
    if [[ "$Batch_state"  -ge 1 ]]; then
        awk -v b=$batchname '
{
    if ($0 ~ "Processing.*" b) {
        s = $2;
        getline;
        e = $2;
        print b " " s " " e;
        exit
    }
}' $fname
    else
        echo $batchname encountered an error
    fi
done

grep -A使用(的较短版本-A是 GNU 扩展，它选择匹配行和后面的行数（默认为 1）：

#!/bin/ksh

fname=$1

for batchname in $(grep -i "Processing batch" $fname | cut -d "'" -f2); do
    Batch_state=`grep -c -i "Batch '$batchname' was successful" $fname`
    printf "%s" $batchname
    if [[ "$Batch_state"  -ge 1 ]]; then
        grep -A1 "Processing batch '$batchname'" $fname | cut -f2 -d" " | fmt
    else
        printf "encountered an error\n"
    fi
done

但这些仍在处理文件3次。请参阅 ChuckCottrill 的回答了解如何完成。

¹它似乎批次不是并行处理的，您想要的实际上是包含该行Processing batch及其之后的行。

Question 2

该命令会将一行内容赋值给x，（同意）

x=awk '/[0-9]_[0-9].*successful/{getline;getline;getline;print}' log filename | awk '{print $2}'
echo "$x"
2341

您可以处理整个文件三次，但您可以扫描一次，然后提取所需的所有内容。

假设您处理整个文件并计算行数。您使用了 awk，但这是一个 Perl 脚本，只需几行即可完成所有操作，

#!/bin/env perl
use strict;
use warnings;

if( $#ARGV < 0 ) { die "no filename"; }
my ($batchname,$S_Time,$E_Time);
my ($success); #batch successul/failed?
my ($num,$pnum); #linenumber, processing line number
my ($fn,$fh,$line,@cols);

foreach $fn (@ARGV)
{
    if( !open($fh,"<$fn") ) { print "error: cannot open $fn\n"; next; }
    $success=0; $num=0; $pnum=-1; #line number
    while($line = <$fh>)
    {
        if( $line =~ /Batch.*\'([\d_]+)\'.*was successful/ )
        {
            $batchname = $1; $success = 1;
        }
        if( $line =~ /Processing batch \'([\d_]+)\'/ )
        {
            $batchname = $1; $pnum = $num;
        }
        if( $pnum > 0 ) {
        my @cols = split(/\s/,$line);
        if($num==$pnum) { $S_Time=$cols[1]; }
        if($num==$pnum+1) { $E_Time=$cols[1]; }
        }
        $num++;
    }
    if( $success ) { print "$batchname\t$S_Time\t$E_Time\n"; }
    else { print "$batchname encountered an error\n"; }
}

从您提供的示例输入/日志文件中：

2013/06/11 13:26:57 <0999> (725102)
Creating batch '12_2013162201.0'...
2013/06/11 13:26:57 <0999> (725102)
Batch '12_2013162201' was successful
2013/06/11 13:26:57 <0999> (725102)
TMR:Child ZERO, 160 Docs 320 Pgs 3874 KByts Tot 0.42 WAL 0.10 WALIO 0.15 IO 0.03 secs
2013/06/11 13:26:57 <0999> (725102)  Processing batch '12_2013162201'
2013/06/11 13:26:57 <0999> (725102)
Total in batch: 160 documents using 4 KBytes

这是输出，

$ ./extractbatch.pl extractbatch.log 
12_2013162201   13:26:57        13:26:57

这是 awk 版本（首先对 Perl 感到抱歉，我对 awk 有点生疏），

#!/bin/awk -f
BEGIN {
    success=0; batchname="none";
    num=0; pnum=-1;
    S_Time="null"; E_Time="null";
}
#/Batch*was successful/ {
/was successful/ {
    batchname = $2; success = 1;
    printf "1: %s\n",batchname;
}
#/Processing batch */ {
/Processing/ {
    batchname = $7; pnum = num;
    printf "2: %s\n",batchname;
}
{
    if(num==pnum) S_Time=$2;
    if(num==pnum+1) E_Time=$2;
    num++;
}
END {
    if( success>0 ) { printf "%s\t%s\t%s\n",batchname,S_Time,E_Time; }
    else { printf "%s encountered an error\n",batchname; }
}

请注意，perl 版本的功能更多。

Answer