☰

根据关键列合并一个文件的行

tag-icon • tag-icon shell scripting
根据关键列合并一个文件的行

我有一个文本文件，其中包含 ~ 分隔符值。前 4 列是关键列，需要根据第 5 列值提取。该文件如下所示：

S1~D1~1~1~abc
S1~D1~1~1~bef
S2~D1~1~2~xyz
S2~D1~1~2~mnp

输出应该是这样的

S1~D1~1~1~abcbef
S2~D1~1~2~xyzmnp

最后一列有时会有换行符，应将其删除，因为它将正确合并两行。

答案1

awk '
  BEGIN{OFS = FS = "~"} 
  {x = $5; NF--; a[$0] = a[$0] x; next} 
  END{for(i in a) print i, a[i]}
' file
S1~D1~1~1~abcbef
S2~D1~1~2~xyzmnp

答案2

如果您可以将 Python 与 Pandas 一起使用：

#!/usr/bin/python3
# combine_by_keys.py
# Take input file with tilde-delimited keys and combine strings with exact key
# match.
#
# ./combine.py myfile.txt
#
import pandas as pd
import re
import sys

filename = sys.argv[1]

# Read keys and strings into lists.
keys = list()                
strings = list()             
with open(filename) as f:
    for line in f.readlines():
        key, string = line.strip().rsplit('~', 1)
        keys.append(key)
        strings.append(string)

print(re.sub(
        r'\ +',  # Pandas separates index and values with spaces.
        '~',     # Replace spaces with tilde.
        pd.DataFrame({'keys':keys,'strings':strings})
                .groupby('keys')['strings']
                .apply(lambda x:''.join(x))
                .to_string(header=False)))

在这样的数据文件上使用它。

$ cat myfile.txt 
S1~D1~1~1~abc
S1~D1~1~1~bef
S2~D1~1~2~xyz
S2~D1~1~2~mnp

$ ./combine_by_keys.py myfile.txt 
S1~D1~1~1~abcbef
S2~D1~1~2~xyzmnp

相关内容