☰

将文本文件重新格式化为 CSV 格式

tag-icon • tag-icon text-processing awk sed csv
将文本文件重新格式化为 CSV 格式

输入样本

0bef-82-46-8a-9a0b.xml "Fruits/Mango Apple /Plum cherry date">1446815.ABC
0bef-82-46-8a-9a0b 5da-0-ba-c1-1a9 "Fruits/Pear Banana/Plum orange mango"
0bef-82-46-8a-9a0b ac-94-4ab-91-23 "Fruits/Pear Banana/Plum orange mango"
0bef-82-46-8a-9a0b 5z-94-ab-92-2f3 "Fruits/Pear Banana/Plum orange mango"

952f-82-46-8a-9a0b.xml "Fruits/Mango"1244115.ABC
3cff-82-46-8a-9a0b.xml "Fruits/Big Mango/Not Sweet ">905499.ABC
6m0k-82-46-8a-9a0b.xml "Fruits/Big Pear/Very Sweet">855499.ABC

17a-42-df-c24.xml "Fruits Market/Big Apple/Sweet "1483415.ABC
17a-42-df-c24 54-ba-4411-9-3d8 "Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24 2da5-0-4a-b1-e89 "Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24 b7-94-4db-92-2f3 "Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24 4d-67c-446-b5-ac "Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24 2-8b-4det-87-769 "Veg/Radish /Radish Carrot Celery Onion"

预期输出 -

0bef-82-46-8a-9a0b.xml,"Fruits/Mango Apple /Plum cherry date",0bef-82-46-8a-9a0b,5da-0-ba-c1-1a9,"Fruits/Pear Banana/Plum orange mango"
0bef-82-46-8a-9a0b.xml,"Fruits/Mango Apple /Plum cherry date",0bef-82-46-8a-9a0b,ac-94-4ab-91-23,"Fruits/Pear Banana/Plum orange mango"
0bef-82-46-8a-9a0b.xml,"Fruits/Mango Apple /Plum cherry date",0bef-82-46-8a-9a0b,5z-94-ab-92-2f3,"Fruits/Pear Banana/Plum orange mango"

952f-82-46-8a-9a0b.xml,"Fruits/Mango",,
3cff-82-46-8a-9a0b.xml,"Fruits/Big Mango/Not Sweet ",,
6m0k-82-46-8a-9a0b.xml,"Fruits/Big Pear/Very Sweet",,


17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,54-ba-4411-9-3d8,"Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,2da5-0-4a-b1-e89,"Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,b7-94-4db-92-2f3,"Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,4d-67c-446-b5-ac,"Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,2-8b-4det-87-769,"Veg/Radish /Radish Carrot Celery Onion"

在输入原始数据中：

  1. 每行中没有前导和尾随空格。
  2. 行之间没有空格。显示的空白旨在使其看起来美观/易于理解。最终输出中也不需要空白。
  3. 几行中缺少符号“>”。这不是一个错字。

您能否指导我如何使用 bash/shell 脚本（sed、awk 等）重新格式化。我迷路了。

答案1

使用awk：

awk '{
  if (sub(/\.xml /, ".xml,")){      # replace `.xml ` with `.xml,`
    if (NR>1 && is_processed != 1){ # xml line was not printed?
       print xml","                 # print previous xml line + `,`
    }
    sub(/>?[0-9]+\.ABC$/, ",") # replace strings `>1446815.ABC` or `1244115.ABC` with `,`
    xml=$0                     # save line in variable `xml`
    is_processed=0             # clear flag
  }
  else {
    if (!NF) next  # skip empty line
    sub(/ /, ",")  # replace 1st ` ` with `,`
    sub(/ /, ",")  # replace 2nd ` ` with `,`
    print xml$0    # print xml line + current line
    is_processed=1 # set flag
  }
}
END {
  # print possible remaining line
  if (is_processed != 1) print xml","
}' filein > fileout

-blockif处理包含该行.xml并将其保存在变量 中xml。 -blockelse处理 xml 行的以下“子项”，并打印 xml 行以及修改后的行，其中前两个空格字符替换为逗号。空行将被跳过。

如果没有“children”，则带有附加逗号的 xml 行将打印在顶部块if（如果行号大于 1）或END块中。

输出 （fileout）：

0bef-82-46-8a-9a0b.xml,"Fruits/Mango Apple /Plum cherry date",0bef-82-46-8a-9a0b,5da-0-ba-c1-1a9,"Fruits/Pear Banana/Plum orange mango"
0bef-82-46-8a-9a0b.xml,"Fruits/Mango Apple /Plum cherry date",0bef-82-46-8a-9a0b,ac-94-4ab-91-23,"Fruits/Pear Banana/Plum orange mango"
0bef-82-46-8a-9a0b.xml,"Fruits/Mango Apple /Plum cherry date",0bef-82-46-8a-9a0b,5z-94-ab-92-2f3,"Fruits/Pear Banana/Plum orange mango"
952f-82-46-8a-9a0b.xml,"Fruits/Mango",,
3cff-82-46-8a-9a0b.xml,"Fruits/Big Mango/Not Sweet ",,
6m0k-82-46-8a-9a0b.xml,"Fruits/Big Pear/Very Sweet",,
17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,54-ba-4411-9-3d8,"Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,2da5-0-4a-b1-e89,"Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,b7-94-4db-92-2f3,"Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,4d-67c-446-b5-ac,"Veg/Radish /Radish Carrot Celery Onion"
17a-42-df-c24.xml,"Fruits Market/Big Apple/Sweet ",17a-42-df-c24,2-8b-4det-87-769,"Veg/Radish /Radish Carrot Celery Onion"

答案2

使用米勒（https://github.com/johnkerl/miller) 和 sed

<input.csv sed -r 's|^(.+")(.?[0-9]+.+)$|\1 "\2"|g' | \
mlr --csv -N --ifs " " put 'if($1=~"xml") {$4=$1;$5=$2}' \
then unsparsify \
then fill-down -f 4,5  \
then count-similar -g 4 \
then filter '($1=~"xml" && $count==1) || ($1!=~"xml" && $count>1)' \
then reorder -f 4,5,1,2,3 \
then put 'if($2=~"Fru"){$1="";$2="";$3=""}' \
then cut -x -f count

你将会拥有

+------------------------+--------------------------------------+--------------------+------------------+----------------------------------------+
| 0bef-82-46-8a-9a0b.xml | Fruits/Mango Apple /Plum cherry date | 0bef-82-46-8a-9a0b | 5da-0-ba-c1-1a9  | Fruits/Pear Banana/Plum orange mango   |
| 0bef-82-46-8a-9a0b.xml | Fruits/Mango Apple /Plum cherry date | 0bef-82-46-8a-9a0b | ac-94-4ab-91-23  | Fruits/Pear Banana/Plum orange mango   |
| 0bef-82-46-8a-9a0b.xml | Fruits/Mango Apple /Plum cherry date | 0bef-82-46-8a-9a0b | 5z-94-ab-92-2f3  | Fruits/Pear Banana/Plum orange mango   |
| 952f-82-46-8a-9a0b.xml | Fruits/Mango                         | -                  | -                | -                                      |
| 3cff-82-46-8a-9a0b.xml | Fruits/Big Mango/Not Sweet           | -                  | -                | -                                      |
| 6m0k-82-46-8a-9a0b.xml | Fruits/Big Pear/Very Sweet           | -                  | -                | -                                      |
| 17a-42-df-c24.xml      | Fruits Market/Big Apple/Sweet        | 17a-42-df-c24      | 54-ba-4411-9-3d8 | Veg/Radish /Radish Carrot Celery Onion |
| 17a-42-df-c24.xml      | Fruits Market/Big Apple/Sweet        | 17a-42-df-c24      | 2da5-0-4a-b1-e89 | Veg/Radish /Radish Carrot Celery Onion |
| 17a-42-df-c24.xml      | Fruits Market/Big Apple/Sweet        | 17a-42-df-c24      | b7-94-4db-92-2f3 | Veg/Radish /Radish Carrot Celery Onion |
| 17a-42-df-c24.xml      | Fruits Market/Big Apple/Sweet        | 17a-42-df-c24      | 4d-67c-446-b5-ac | Veg/Radish /Radish Carrot Celery Onion |
| 17a-42-df-c24.xml      | Fruits Market/Big Apple/Sweet        | 17a-42-df-c24      | 2-8b-4det-87-769 | Veg/Radish /Radish Carrot Celery Onion |
+------------------------+--------------------------------------+--------------------+------------------+----------------------------------------+

笔记：作为输入，我使用了没有任何空行的 CSV

相关内容