☰

更快的 awk 脚本来获取我们想要的子字符串/字符串

tag-icon • tag-icon shell-script awk performance
更快的 awk 脚本来获取我们想要的子字符串/字符串 
ORDER EVENT .........[] [] ... so many other tags... [Account<25106>=ACCT1] [Destination...] .. so many other tags.

我目前正在尝试获得这样的帐户。我尝试在 awk 中使用 match，但似乎速度较慢。除了下面更快的方法之外，您还能建议其他吗？

j = index($0, "<25106>=");
account=substr($0, j + accountTagLength);
account=substr(account,1,index(account, "]") - 1);

帐户不是第二个字段，并且字段位置可能有所不同。

时间：

bash-3.2$ time head -1000000 temp.log | awk -F'<25106>=' '{print $2}' | sed -e 's/].*//' > /dev/null

real    0m2.410s
user    0m2.782s
sys     0m0.319s
bash-3.2$ time head -1000000 temp.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.690s
user    0m1.737s
sys     0m0.448s
bash-3.2$ time head -1000000 temp.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.588s
user    0m1.733s
sys     0m0.179s
bash-3.2$ time head -1000000 temp.log | awk -F'<25106>=' '{print $2}' | sed -e 's/].*//' > /dev/null                               
real    0m2.384s
user    0m2.762s
sys     0m0.272s
bash-3.2$ time head -1000000 temp.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.703s
user    0m1.709s
sys     0m0.484s

bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | gawk 'match($0, /<25106>=([^]]+)/, ary) {account = ary[1]}'

real    0m3.449s
user    0m3.661s
sys     0m0.290s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | gawk 'match($0, /<25106>=([^]]+)/, ary) {account = ary[1]}'

real    0m3.410s
user    0m3.551s
sys     0m0.236s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | gawk 'match($0, /<25106>=([^]]+)/, ary) {account = ary[1]}'

real    0m3.361s
user    0m3.487s
sys     0m0.286s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.626s
user    0m1.831s
sys     0m0.263s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk -F '<25106>=' '{split($2, ary, /\]/); account = ary[1]}'

real    0m2.721s
user    0m2.808s
sys     0m0.265s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk -F '<25106>=' '{split($2, ary, /\]/); account = ary[1]}'

real    0m2.787s
user    0m2.863s
sys     0m0.516s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk -F '<25106>=' '{split($2, ary, /\]/); account = ary[1]}'

real    0m2.724s
user    0m2.882s
sys     0m0.278s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.576s
user    0m1.748s
sys     0m0.235s

bash-3.2$ time head -100000 ORDER_EVENTS_CHAS_20120224.log | grep -oE '<25106>=([A-Za-z0-9]*)+' | cut -d= -f2 > /dev/null                                     
real    0m2.098s
user    0m2.131s
sys     0m0.033s
bash-3.2$ time head -100000 ORDER_EVENTS_CHAS_20120224.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); print substr(account,1,index(account, "]") - 1);} }' > /dev/null

real    0m0.253s
user    0m0.275s
sys     0m0.040s
bash-3.2$ time head -100000 ORDER_EVENTS_CHAS_20120224.log | grep -oE '<25106>=([A-Za-z0-9]*)+' | cut -d= -f2 > /dev/null                                     
real    0m2.070s
user    0m2.105s
sys     0m0.034s
bash-3.2$ time head -100000 ORDER_EVENTS_CHAS_20120224.log | grep -oE '<25106>=([A-Za-z0-9]*)+' > /dev/null

real    0m2.065s
user    0m2.090s
sys     0m0.037s
    bash-3.2$ time head -1000000 ORDER_EVENTS_CHAS_20120228.log | awk -F'<25106>=' '{ substr($2,0,index($2,"]")-1);}'

real    0m3.426s
user    0m3.637s
sys     0m0.412s
bash-3.2$ time head -1000000 ORDER_EVENTS_CHAS_20120228.log | awk -F'<25106>=' '{ substr($2,0,index($2,"]")-1);}'

real    0m3.463s
user    0m3.603s
sys     0m0.408s
bash-3.2$ time head -1000000 ORDER_EVENTS_CHAS_20120228.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m2.247s
user    0m2.307s
sys     0m0.649s

答案1

尝试这个：

awk -F'<25106>=' '{print substr($2,0,index($2,"]")-1);}'

不使用正则表达式，只是严格的字符串操作。

答案2

如果您有 GNU awk ( gawk)，您可以使用match()带有捕获括号的函数：

gawk 'match($0, /<25106>=([^]]+)/, ary) {account = ary[1]}'

或者，您可以使用复杂的字段分隔符：

awk -F '<25106>=' '{split($2, ary, /\]/); account = ary[1]}'

答案3

如果您只打印这个号码，您可以尝试以下操作：

echo "ORDER EVENT ......... [Account<25106>=ACCT1]" | awk -F'<25106>=' '{print $2}' | sed -e 's/].*//'

编辑：仅 sed 解决方案：

echo "ORDER EVENT ......... [Account<25106>=ACCT1]" | sed -e 's/.*25106>=//' -e 's/].*//'

编辑2：

awk '{if (split($0, a, "25106>=") > 1) {print substr(a[2], 0, index(a[2], "]")-1)} }'

相关内容