☰

使用 Grep 查找错误消息周围的块

tag-icon • tag-icon linux bash logging grep
使用 Grep 查找错误消息周围的块

我想 grep 特定错误消息周围的日志块。假设我知道我的日志如下所示：

[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)

我对整个区块都很感兴趣。有没有办法让我的 grep 以错误为中心FATAL，并抓取到Started GET和 到Completed 200 OK？

我知道这grep -B 123 -A 456 "FATAL" file.txt可以解决问题，但这需要我提前知道FATAL第一行（123）和最后一行（456）之间的距离[1]。

[1]http://unixhelp.ed.ac.uk/CGI/man-cgi?grep

答案1

恐怕 grep 没有这样的功能。perl 和/或 python 可能有。我一时兴起（没有花太多心思）就想出了以下 awk 组合。

 awk '/Started GET/{print ""};1' blurble | awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FATAL/{print $0}'

给定一个从你的代码片段（x4）中编造的日志块和一些额外的“其他日志”行以及两个“FATAL”行被删除，我得到了两个打印段：

[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)
[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)
[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)
[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)

对该文件（我称之为 blurble）运行两个 awk 会得到以下输出：

[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)

[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)

答案2

我有一个可能有帮助的脚本：https://github.com/jakl/blocks

./blocks.pl --delim '开始 GET|完成 200 OK' --term 'FATAL'

它将日志分成以 --delim 开头或结尾的块，并打印包含 --term 的块

答案3

这是使用 sed 的一行代码

sed -n '/Started/,/Completed/H;/Completed/{z;x;/FATAL/p}' log

一行代码展开解释

# do not print lines unless explicitly told to do so
sed -n '

  # put everything between Started and Completed in hold space
  /Started/,/Completed/ H;

  # if at Completed
  /Completed/ {

    # empty the pattern space
    z;

    # switch hold space with pattern space
    x;

    # if FATAL in pattern space then print
    /FATAL/p

  }

' log

请注意，这将在每个块之间打印一个空行。空行来自组合z;x;。z清空模式空间。x将保持空间与模式空间切换。这意味着z;x;在保持空间中有一个空行。如果在与保持空间切换之前没有清空模式空间，则下一个块中将出现前一个块的最后一行。我试图解决这个问题以防止出现空行，但这使代码变得更加复杂。

如果空行让您感到困扰，您可以将其过滤掉...使用另一个 sed 命令。

sed -n '/Started/,/Completed/H;/Completed/{z;x;/FATAL/p}' log | sed '/^$/d'

或者你可以放置一个标记，使该命令的输出几乎像 grep

sed -n '/Started/,/Completed/H;/Completed/{z;x;/FATAL/p}' log | sed 's/^$/--/'

这是我想出的最简单的解决方案，可以消除空行

sed -n '

  # replace hold space with pattern space
  /Started/ h

  /Started/,/Completed/ {

    # skip the first line because it is already in hold space
    /Started/ n

    # append pattern space to hold space
    H
  }

  /Completed/ {

    # switch hold space with pattern space
    x

    # if FATAL in pattern space then print
    /FATAL/ p
  }

' log

与强制性的一行相同

sed -n '/Started/h;/Started/,/Completed/{/Started/n;H};/Completed/{x;/FATAL/p}' log

相关内容