如何使用 awk 使用文本模式分割文件

如何使用 awk 使用文本模式分割文件

我有一个 2Gb 的文件。它有一个标头和很多很多“事件”结构。这就是开头的样子:

<run example>
<header>
 5
   This is header                                                   
</header>

<event  =             22>
<evhead>
 8
       3       1       2       0       0       0       0       0
       0       0       1       0       1      30       0       1
       4       1       4       3       1       0       1       0
       0       0       0       0       0       1       1       8
       0       1       0       2       1       5       2       0
       2       1       3       7       3       1       1       0
       1       0   10100       2       3       1       5       1
       1       5       1       7       2       3       2       2
</evhead>
 0 97
           3      11       0       0       3       4  1.94791176123E-14  0.00000000000E+00 -2.75000000000E+01  2.75000000047E+01  5.10000000000E-04
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3    2212       0       0       5       0 -1.94791176123E-14  0.00000000000E+00  9.20000000000E+02  9.20000478451E+02  9.38270000000E-01
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3      11       1       0       0       0  4.63012694434E+00  2.62561831936E+00 -2.31855757639E+01  2.37887130977E+01  5.10000000000E-04
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3      22       1       0       0       0 -4.63012694434E+00 -2.62561831936E+00 -4.31442423592E+00  3.71128690719E+00 -5.75956188088E+00
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3    2212       2       0       0       0 -2.16995636615E-14 -1.11022302463E-15  9.20000000000E+02  9.20000478451E+02  9.38270000000E-01
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3      22       4       0       0       0 -4.60626572550E+00 -2.61208727495E+00 -2.23619853289E+00  5.74815342040E+00  0.00000000000E+00
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
</event>

整个文件包含 97000 个这样的“事件”块。因此,我想将此文件拆分为 10 个文件,每个文件都包含标头和 10 000 个“事件”块。所有块的索引都不同(它们是随机的)。最后一个文件当然只包含 7000 个块。

我已经尝试了来自堆栈的多个指令,如下所示:https://stackoverflow.com/questions/8544197/splitting-a-file-in-linux-based-on-content https://stackoverflow.com/questions/8544197/splitting-a-file-in-linux-based-on-content 但没有对我有用。

下面是用于任何测试的文件的更大示例(要下载的文件):

<run example>
<header>
 5
   header                                                     
</header>

<event  =             22>
<evhead>
 8
       3       1       2       0       0       0       0       0
       0       0       1       0       1      30       0       1
       4       1       4       3       1       0       1       0
</evhead>
 0 97
           3      11       0       0       3       4  1.94791176123E-14  0.00000000000E+00 -2.75000000000E+01  2.75000000047E+01  5.10000000000E-04
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3    2212       0       0       5       0 -1.94791176123E-14  0.00000000000E+00  9.20000000000E+02  9.20000478451E+02  9.38270000000E-01
</event>

<event  =             26>
<evhead>
 8
       3       1       2       0       0       0       0       0
       0       0       1       0       1      30       0       1
       4       1       4       3       1       0       1       0
</evhead>
 0 52
           3      11       0       0       3       4  1.94791176123E-14  0.00000000000E+00 -2.75000000000E+01  2.75000000047E+01  5.10000000000E-04
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3    2212       0       0       5       0 -1.94791176123E-14  0.00000000000E+00  9.20000000000E+02  9.20000478451E+02  9.38270000000E-01
</event>

<event  =             31>
<evhead>
 8
       3       1       2       0       0       0       0       0
       0       0       1       0       1      30       0       1
       4       1       4       3       1       0       1       0
       0       0       0       0       0       1       1       8
</evhead>
 0 92
           3      11       0       0       3       4  1.94791176123E-14  0.00000000000E+00 -2.75000000000E+01  2.75000000047E+01  5.10000000000E-04
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3    2212       0       0       5       0 -1.94791176123E-14  0.00000000000E+00  9.20000000000E+02  9.20000478451E+02  9.38270000000E-01
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3      11       1       0       0       0  4.39003604933E+00  4.97037860337E+00 -2.04926313413E+01  2.15389187176E+01  5.10000000000E-04
</event>

<event  =             37>
<evhead>
 8
       3       1       2       0       0       0       0       0
       0       0       1       0       1      30       0       1
</evhead>
 0 77
           3      11       0       0       3       4  1.94791176123E-14  0.00000000000E+00 -2.75000000000E+01  2.75000000047E+01  5.10000000000E-04
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3    2212       0       0       5       0 -1.94791176123E-14  0.00000000000E+00  9.20000000000E+02  9.20000478451E+02  9.38270000000E-01
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3      11       1       0       0       0  7.91768942174E+00  3.75815788575E+00 -2.09569980000E+01  2.27158385693E+01  5.10000000000E-04
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
</event>

<event  =             41>
<evhead>
 8
       3       1       2       0       0       0       0       0
       0       0       1       0       1      30       0       1
       4       1       4       3       1       0       1       0
</evhead>
 0 122
           3      11       0       0       3       4  1.94791176123E-14  0.00000000000E+00 -2.75000000000E+01  2.75000000047E+01  5.10000000000E-04
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3    2212       0       0       5       0 -1.94791176123E-14  0.00000000000E+00  9.20000000000E+02  9.20000478451E+02  9.38270000000E-01
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
           3      11       1       0       0       0 -3.63469912393E+00  3.95372353695E+00 -1.62133507727E+01  1.70796870892E+01  5.10000000000E-04
                                                      0.00000000000E+00  0.00000000000E+00  0.00000000000E+00  0.00000000000E+00
</event>

答案1

使用 GNU awk

BEGIN { fname = "/dev/null" }
/<header>/,/<\/header>/ { hdr = hdr $0 "\n"; next }
/^<event / {
    events++
    if(events % 10000 == 1) {
        if(files++) close(fname)
        fname = sprintf("file%02d.txt", files)
        print hdr >fname
    }        
}            
{ print >>fname }

要运行它:将其写入文件script.awk,然后运行:

gawk -f script.awk file.txt

相关内容