☰

Bash 时差

tag-icon • tag-icon linux bash shell-script shell date
Bash 时差

我有一个日志文件，我需要在两个作业之间进行一些计算以找到时间差。

在我的日志文件中，我应该计算作业之间的时间incoming request，candidate list sent for以下日志中的时间为 2 秒；

2019-08-23 00:05:27 42303: incoming request: 1dd5.073f.5d5f0397 (156074 bytes)
2019-08-23 00:05:27 42303: store file: /papillon1/vrf/rq/1dd5.073f.5d5f0397.rq, len: 156074
2019-08-23 00:05:27 42303: registering process 42605 left to finish it's work
2019-08-23 00:05:27 42605: making search request for: 1dd5.073f.5d5f0397
2019-08-23 00:05:27 42605: 1dd5.073f.5d5f0397 is request for search by fingers
2019-08-23 00:05:27 42605: portions: 156
2019-08-23 00:05:27 42605: request pattern version 8
2019-08-23 00:05:27 42605: fingers mask: 1111111111; quality mask: 3011031110
2019-08-23 00:05:27 42605: saving request file: /papillon1/vrf/rqm/1dd5.073f.5d5f0397.rqm~
2019-08-23 00:05:27 42605: renaming request file: /papillon1/vrf/rqm/1dd5.073f.5d5f0397.rqm~ -
> /papillon1/vrf/rqm/1dd5.073f.5d5f0397.rqm
2019-08-23 00:05:27 42605: request file saved /papillon1/vrf/rqm/1dd5.073f.5d5f0397.rqm
2019-08-23 00:05:27 42605: request is in queue: 1dd5.073f.5d5f0397
2019-08-23 00:05:27 42605: request 1dd5.073f.5d5f0397 registering time: 663 msec
2019-08-23 00:05:28 42303: waiting for 42605 to be finished
2019-08-23 00:05:28 42303: 42605 finished; waiting time: 0 ms
2019-08-23 00:05:29 43188: candidate list for 1dd5.073f.5d5f0397; 2 records
2019-08-23 00:05:29 43188: candidate list file size: 381
2019-08-23 00:05:29 43188: candidate list sent for: 1dd5.073f.5d5f0397

因此，我创建了一个脚本来读取日志文件中的所有映射，并在该映射的 2 个作业之间进行计算。上面的日志中1dd5.073f.5d5f0397以一张地图为例。

这也是我的完整脚本；

#!/bin/bash

for i in $(grep 'incoming request:' a8.svrf.ear | sed 's/^.*: //' | awk -F'[ ]' '{print $1}')
do
    var0=$i

    TIME1=$(grep 'incoming request:' a8.svrf.ear | awk -F'[ ]' '{print $2}')

    TIME2=$(grep 'candidate list sent for:' a8.svrf.ear | grep "$var0" | awk -F'[ ]' '{print $2}')

    SEC1=$(date +%s -d "${TIME1}")

    SEC2=$(date +%s -d "${TIME2}")

    DIFFSEC=$(expr "${SEC2}" - "${SEC1}")

    echo Map "${var0}" >> /home/st/anil/test.txt

    echo Start "${TIME1}" >> /home/st/anil/test.txt
    echo Finish "${TIME2}" >> /home/st/anil/test.txt

    echo Took "${DIFFSEC}" seconds >> /home/st/anil/test.txt

    echo Took $(date +%H:%M:%S -ud @"${DIFFSEC}") >>/home/st/anil/test.txt

    echo =========================================================================


done

在我的代码中，我基本上尝试使用 for 循环获取所有地图，对于每个地图，我尝试为 2 个作业获取时间并找到时间差，但我的输出是意外的；

=========================================================================
...
34\n11:33:42\n11:33:42\n11:33:47’
expr: non-integer argument
date: invalid date ‘@’
=========================================================================
date: invalid date ‘00:01:37\n00:05:27\n00:09:49\n00:11:18\n00:12:02\n00:12:28\n00:12:52\n00:13:24\n00:15:10\n00:16:
...

另外，在输出 txt 文件中，我对每个地图都会遇到类似的情况；

Map 1dd5.0721.5d5f02b1
Start 00:01:37
00:05:27
00:09:49
00:11:18
00:12:02
00:12:28
00:12:52
00:13:24
00:15:10
00:16:05
00:22:36
00:23:14
00:23:44
00:24:15
00:25:26
00:26:07
00:27:04
00:27:34
...
Finish 
Took  seconds
Took
=========================================================================

如果你们指出我的错误，我会很高兴。谢谢

答案1

#!/usr/bin/perl

use strict;

use Date::Parse;
use Date::Format;

# hash arrays to keep the start and ending times for each request id.
my %start = ();
my %end = ();

# the @order array is used to preserve the order that we saw request ids,
# so we can print them out in the same order.
my @order;

while(<>) {
  # skip lines we're not interested in
  next unless (m/incoming request:|candidate list sent for:/);
  chomp;  # strip trailing linefeed

  # split input line into array @F on white-space separated fields.
  my @F = split;
  # and get the timestamp of the log entry
  my $timestamp = join(" ", @F[0..1]);

  if (m/incoming request:/) {
    my $reqid = $F[5];
    if (!defined($start{$reqid})) { push @order, $reqid };

    $start{$reqid} = $timestamp;

  } elsif (m/candidate list sent for:/) {
    my $reqid = $F[7];
    $end{$reqid} = $timestamp;
  };

};

foreach my $reqid (@order) {
    my $seconds = str2time($end{$reqid}) - str2time($start{$reqid});
    my $hms = time2str("%H:%M:%S", $seconds, "0");
    my $s = time2str("%H:%M:%S", str2time($start{$reqid}));
    my $e = time2str("%H:%M:%S", str2time($end{$reqid}));

    print <<__EOF__
Map $reqid
Start $s
Finish $e
Took $seconds seconds
Took $hms
=========================================================================

__EOF__
}

将其另存为，例如，anil.pl并使其可执行chmod +x anil.pl。

样本输入的输出为：

$ ./anil.pl a8.svrf.ear 
Map 1dd5.073f.5d5f0397
Start 00:05:27
Finish 00:05:29
Took 2 seconds
Took 00:00:02
=========================================================================

相同的算法可以在 bash 中实现，但是（IMO）它更难阅读和理解，并且 bash 数组变量使用起来很痛苦，因为它们需要如何引用。而且它的运行速度会慢很多：

例如：

#!/bin/bash

declare -A start end
declare -a order

while read -r -a F ; do

  [ "${F[3]} ${F[4]}" != "incoming request:" ] \
    && [ "${F[3]} ${F[6]}" != "candidate for:" ] \
    && continue

  ts="${F[0]} ${F[1]}"

  if [ "${F[3]}" = "incoming" ] ; then
    reqid="${F[5]}"
    [ -v start[$reqid] ] || order+=( "$reqid" )
    start["$reqid"]="$ts"

  elif [ "${F[3]}" = "candidate" ] ; then
    reqid="${F[7]}"
    end["$reqid"]="$ts"
  fi

done <(cat "$@")

for reqid in "${order[@]}"; do
  start=$(date -d "${start[$reqid]}" +%s)
  end=$(date -d "${end[$reqid]}" +%s)
  seconds=$(( $end - $start ))
  hms=$(TZ=UTC date -d "@$seconds" "+%H:%M:%S")
  s=$(date -d "${start[$reqid]}" "+%H:%M:%S")
  e=$(date -d "${end[$reqid]}" "+%H:%M:%S")

cat <<__EOF__
Map $reqid
Start $s
Finish $e
Took $seconds seconds
Took $hms
=========================================================================

__EOF__

done

注意：这需要 bash v4.3 或更高版本，以便-v测试操作员检查变量是否存在。

答案2

我无法重现你的错误。我进行了更改以参数化您的输出文件，因为我没有您的 /home/st... 如前所述：

#!/bin/bash

for i in $(grep 'incoming request:' a8.svrf.ear | sed 's/^.*: //' | awk -F'[ ]' '{print $1}')
do
    var0=$i

    TIME1=$(grep 'incoming request:' a8.svrf.ear | awk -F'[ ]' '{print $2}')

    TIME2=$(grep 'candidate list sent for:' a8.svrf.ear | grep "$var0" | awk -F'[ ]' '{print $2}')

    SEC1=$(date +%s -d "${TIME1}")

    SEC2=$(date +%s -d "${TIME2}")

    DIFFSEC=$(expr "${SEC2}" - "${SEC1}")

        OUT=/home/st/anil/test.txt
        OUT=t1
        rm -f $OUT
    # echo Map "${var0}" >> /home/st/anil/test.txt
    echo Map "${var0}" >> $OUT

    # echo Start "${TIME1}" >> /home/st/anil/test.txt
    echo Start "${TIME1}" >> $OUT
    # echo Finish "${TIME2}" >> /home/st/anil/test.txt
    echo Finish "${TIME2}" >> $OUT

    # echo Took "${DIFFSEC}" seconds >> /home/st/anil/test.txt
    echo Took "${DIFFSEC}" seconds >> $OUT

    # echo Took $(date +%H:%M:%S -ud @"${DIFFSEC}") >>/home/st/anil/test.txt
    echo Took $(date +%H:%M:%S -ud @"${DIFFSEC}") >> $OUT

    echo =========================================================================


done

输出文件 t1 包含：

Map 1dd5.073f.5d5f0397
Start 00:05:27
Finish 00:05:29
Took 2 seconds
Took 00:00:02

这是在这样的系统上：

OS, ker|rel, machine: Linux, 3.16.0-7-amd64, x86_64
Distribution        : Debian 8.11 (jessie) 
bash GNU bash 4.3.30
date (GNU coreutils) 8.23

要设置您的文件，只需交换 2 个 OUT 分配。

祝你好运...干杯，drl

相关内容