☰

使用 awk 进行格式化

tag-icon • tag-icon command-line bash sed awk
使用 awk 进行格式化

我有这样的文字：

Core/Dev
Core/master
FrontEnd/Dev
FrontEnd/DoubleClick
FrontEnd/HEAD -> FrontEnd/master
FrontEnd/master
Infrastructure/Dev
Infrastructure/master
Software/Dev
Software/Control
Software/master

我想实现这个目标：

Core:Dev,master
FrontEnd:Dev,DoubleClick,master
Infrastructure:Dev,master
Software:Dev,Control,master

请注意，所有以相同开头的行都会被合并，并且值以逗号分隔，

HEAD并且将忽略带有 的行。

最好的办法是什么？sed或者awk？怎么做？谢谢。

答案1

我建议在这种情况下使用 awk。创建此文件（名称filter-au.awk：）：

#!/usr/bin/awk
BEGIN {
   # first field on output list. Ex: FrontEnd
   last_reg="xxxxxxxxxxxx";
   # second field on output list. Ex: Dev,DoubleClick,master
   parts="";
   # number of parts in second field. 
   # Ex: from previous case. num_parts_on_reg=3
   num_parts_on_reg=0;
   # flag to signalize a new part has to be included in
   # string parts
   flag_part=0;
}
{
    # not first line?
    if ( NR > 1 ) {
       # $1 field was changed? If yes, print register
       if ( $1 != last_reg ) {
          print last_reg ":" parts;
          num_parts_on_reg=0;
          parts="";
       }

       # fill string parts
       if ( num_parts_on_reg == 0  ) { 
            # first part
            parts=$2;
            num_parts_on_reg++;
       }
       else {
         # verify if $2 field has included in parts before
         split(parts, arr_values, ",", arr_seps)
         flag_part=0;
         for (p in arr_values) {
            if ( p == $2 ) {
               flag_part=1;
               break;
            }
         }
         # if not, include $2 in parts string
         if ( flag_part == 0 ) {
            parts=parts "," $2
            num_parts_on_reg++;
         }
       }
    } else {
       # first register
       parts=$2;
       num_parts_on_reg++;
    }
    last_reg=$1;
}
END {
   # print last register
   if (  num_parts_on_reg > 0 ) {
      print last_reg ":" parts;
   }
}

然后运行以下命令：

grep -v 'HEAD' input.txt | awk -F "/" -f filter-au.awk

input.txt 是一个包含您发布的内容的文件。似乎有效：

Core:Dev,master
FrontEnd:Dev,DoubleClick,master
Infrastructure:Dev,master
Software:Dev,Control,master

如果FrontEnd/HEAD -> FrontEnd/master被解释为两行，则创建另一个文件（名称filter-au2.awk：）：

#!/usr/bin/awk
{
   if ( NF = 1 ) { print $0; }
   else {
      # split "->" lines in two
      print $1;
      print $2;
   }
}

并运行以下管道命令：

cat input-awk.txt | awk -F "->" -f filter-au2.awk | awk -F "/" -f filter-au.awk

输出为：

Core:Dev,master
FrontEnd:Dev,DoubleClick,HEAD ,master
Infrastructure:Dev,master
Software:Dev,Control,master

答案2

您可以使用 Bash 遍历数据文件，然后使用awk它来解析它。类似这样的操作应该可以帮助您入门（so_ubuntu_20160706.txt上面的文件在哪里）：

#!/bin/bash

FILENAME="/home/aploetz/scripts/so_ubuntu_20160706.txt"
VALUE_LIST=""
LAST_KEY=""
FIRST=1

while read -r LINE
do
  #KEY=$(echo $LINE | awk -F '/' '{print $1}')
  #VALUE=$(echo $LINE | awk -F '/' '{print $2}')
  #Use this line to set KEY and VALUE from awk on a single line
  eval $(echo $LINE | awk -F '/' '{KEY = $1; VALUE = $2} END {print "KEY=\""KEY"\";VALUE=\""VALUE"\""}')
  HEAD=$(echo $LINE | grep -v HEAD)

  if [[ "" != "$HEAD" ]]
  then
    if [[ $FIRST -eq 0 ]]
    then
      if [[ $KEY != $LASTKEY ]]
      then
        echo "$LASTKEY:$VALUE_LIST"
        VALUE_LIST=$VALUE
      else
        VALUE_LIST="$VALUE_LIST,$VALUE"
      fi
    else
      FIRST=0
      VALUE_LIST=$VALUE
    fi
  fi

  LASTKEY=$KEY
done < "$FILENAME"

#output last read key and values
echo "$KEY:$VALUE_LIST"

输出结果如下：

Core:Dev,master
FrontEnd:Dev,DoubleClick,master
Infrastructure:Dev,master
Software:Dev,Control,master

我正在做一个grep -v HEAD，然后测试一个空字符串来捕获您不想要的条目。

答案3

和awk

awk -F\/ '
  /HEAD/ {next} # skip lines matching HEAD
  $1 in a {
    a[$1] = a[$1]","$2; next
  }
  {
    if (last in a) print a[last]
    delete a
    a[$1] = $1":"$2
    last = $1
  }
  END {
    print a[last]
  }' input

如果你不介意将整个文件加载到内存中，并且不关心顺序（或者不介意之后重新排序），一个简单的方法可能是

awk -F\/ '
  /HEAD/ {next} # skip lines matching HEAD
  {
    a[$1] = a[$1] == "" ? $1":"$2 : a[$1]","$2
    next;   
  }
  END {
    for (i in a) print a[i]
  }' input | sort

或者使用 GNU awk 的现代版本

gawk -F\/ '
  /HEAD/ {next} # skip lines matching HEAD
  {
    a[$1] = a[$1] == "" ? $1":"$2 : a[$1]","$2
    next;   
  }
  END {
    PROCINFO["sorted_in"] = "@ind_str_asc"
    for (i in a) print a[i]
  }' input

答案4

得到答案了！感谢大家的时间

awk '
    # split remote and branch
    {
        remote = substr($1, 0, index($1, "/") - 1)
        branch = substr($1, index($1, "/") + 1)
    }

    # eliminate HEAD reference
    branch == "HEAD" { next }

    # new remote found
    remote != lastRemote {
        # output remote name
        printf "%s%s:", lastRemote ? "\n" : "", remote
        lastRemote = remote
        # do not output next comma
        firstBranch = 1
    }

    # output comma between branches
    !firstBranch { printf "," }
    firstBranch { firstBranch = 0 }

    # output branch name
    { printf branch }

    # final linebreak
    END { print "" }
'

相关内容