☰

通过修改重新排序管道分隔字段

tag-icon • tag-icon linux text-processing awk sed regular-expression
通过修改重新排序管道分隔字段

我有一个大约 8k 行的 Linux 文本文件，格式如下

|f_name:x|l_name:x|address:x x|city:x|state:x|zip:x|country:x|ordernumber:x|code:x|downloaded:x|exp:09/2017|ip:x.x.x.x|

我想将其排序为以下格式：

ordernumber:x,exp:09/2017,code:x,f_name:x,l_name:x,address:x x,city:x,state:x,zip:x,country:x,ip:x.x.x.x

笔记

文本中的一些数据与该字段存在问题|address:x x|

它可以像这意味着结尾之前|address:x x |有一个；我想删除输出中的一个或多个空格。space|

并且 ; 领域的数据存在问题|exp:09/2017|。显示的数据如下|exp:9/2017|，我想添加0以防月份是单个数字，因此它将出现09/2017在输出中。

请注意，年份可能不同。

例子：

|f_name:x|l_name:x|address:x x |city:x|state:x|zip:x|country:x|ordernumber:x|code:x|downloaded:x|exp:9/2017|ip:x.x.x.x|

预期输出：

ordernumber:x,exp:09/2017,code:x,f_name:x,l_name:x,address:x x,city:x,state:x,zip:x,country:x,ip:x.x.x.x**

答案1

GNUawk解决方案：

awk '{ 
         split($12, a, /[/:]/); 
         if (length(a[2]) == 1) $12=sprintf("%s:%02d/%d", a[1], a[2], a[3]);
         sub(/ *$/, "", $4);
         print $9, $12, $10, $2, $3, $4, $5, $6, $7, $8, $13 
     }' FS='|' OFS=',' file

输出：

ordernumber:x,exp:09/2017,code:x,f_name:x,l_name:x,address:x x,city:x,state:x,zip:x,country:x,ip:x.x.x.x

答案2

概括

我编写了一个 Awk 脚本、一个 Python 脚本和一个 Bash 脚本，每个脚本都应该可以解决您的问题。它们都产生相同的输出。

这是示例数据（取自您的问题并放入文件中data.csv）：

|f_name:x|l_name:x|address:x x|city:x|state:x|zip:x|country:x|ordernumber:x|code:x|downloaded:x|exp:09/2017|ip:x.x.x.x|

这是运行脚本的输出：

ordernumber:x,exp:09/2017,code:x,f_name:x,l_name:x,address:x x,city:x,state:x,zip:x,country:x,ip:x.x.x.x

awk

这是一个awk脚本：

#!/usr/bin/env awk
# transformcsv.awk

# Set the input field-separator (FS) and the output field-separator (OFS)
BEGIN{
    FS="|";
    OFS=",";
}

# Skip empty lines
/^\s*$/ {next;}

# Print lines with the fields reordered as desired
{
   print $9,$12,$10,$2,$3,$4,$5,$6,$7,$8,$13
}

以下是运行它的方法：

awk -f transformcsv.awk data.csv

您也可以将其作为单行代码运行：

awk 'BEGIN{FS="|";OFS=",";}/^\s*$/ {next;}{print $9,$12,$10,$2,$3,$4,$5,$6,$7,$8,$13}' data.csv

Python

这是 Python 脚本：

#!/usr/bin/env python
# -*- coding: ascii -*-
"""transformcsv.py"""

import sys
import csv

# Make a list with the field names in their input order
# NOTE: We padding colums because each row begins
#       and ends with the delimiter `|`
fieldnames = (
    "padding_1",
    "f_name", "l_name", "address", "city", "state", "zip",
    "country", "ordernumber", "code", "downloaded", "exp", "ip",
    "padding_2"
)

# Make a list with the field names in their output order
reordered_fieldnames = (
    "ordernumber", "exp", "code", "f_name", "l_name",
    "address", "city", "state", "zip", "country", "ip"
)

# Read each input row and print out the reordered row
with open(sys.argv[1]) as csvfile:
    reader = csv.DictReader(csvfile, fieldnames=fieldnames, delimiter='|')
    for row in reader:
        print(','.join([row[field] for field in reordered_fieldnames]))

以下是运行该脚本的方法：

python transformcsv.py data.csv

重击

笔记：这可能会是非常大文件速度慢。你可能不应该使用这个 - 我只是为了好玩才包含它。

这是 Bash shell 脚本：

#!/usr/bin/env bash
# transformcsv.sh

while read LINE; do
    if [[ -n "${LINE}" ]]; then

    # Extract the field values
    f_name="$(echo "${LINE}" | cut -d'|' -f2)"
    l_name="$(echo "${LINE}" | cut -d'|' -f3)"
    address="$(echo "${LINE}" | cut -d'|' -f4)"
    city="$(echo "${LINE}" | cut -d'|' -f5)"
    state="$(echo "${LINE}" | cut -d'|' -f6)"
    zip="$(echo "${LINE}" | cut -d'|' -f7)"
    country="$(echo "${LINE}" | cut -d'|' -f8)"
    ordernumber="$(echo "${LINE}" | cut -d'|' -f9)"
    code="$(echo "${LINE}" | cut -d'|' -f10)"
    downloaded="$(echo "${LINE}" | cut -d'|' -f11)"
    exp="$(echo "${LINE}" | cut -d'|' -f12)"
    ip="$(echo "${LINE}" | cut -d'|' -f13)"

    # Output the reordered row
    printf \
        "%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s\n" \
        "${ordernumber}" "${exp}" "${code}" "${f_name}" "${l_name}" \
        "${address}" "${city}" "${state}" "${zip}" "${country}" "${ip}"

    fi
done < "$1"

以下是运行它的方法：

bash transformcsv.sh data.csv

相关内容