我在解析 .csv 文件的脚本时遇到一些问题
有关 .csv 的信息:
#1,13/8/2020,[email protected],[email protected],,Subject,"Dear Dude,
Information have been updated. Please login to APP to review by 30th August 2020. Thank you.
Best regards,
Mr. Mack",Pending,13/8/2020 12:35
这是信息应该如何划分的: 消息 ID、日期、收件人、抄送、密件抄送、主题、正文、状态、时间戳
MsgID=#1
Date=13/8/2020
[email protected]
[email protected]
BCC=
Subject=Subject
Body=Dear Dude,
Information have been updated. Please login to APP to review by 30th August 2020. Thank you.
Best regards,
Mr. Mack
Status=Pending
Timestamp=13/8/2020 12:35
我遇到的问题与正文部分有关,原因有两个:当我尝试运行脚本时文本中的“,”会中断,另一个原因是文本中的换行符。
这是我正在做的脚本:
#!/bin/bash
export TIMESTAMP="$( date '+%d/%m/%Y %H:%M:%S' )"
INPUT=/tmp/test.csv
OUTPUT=/tmp/test.csv.out
OLDIFS=$IFS
IFS=','
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read msgid dat to cc bcc subject body status timesta
do
echo "MSG ID : $msgid"
echo "Date : $dat"
echo "To : $to"
echo "CC : $cc"
echo "BCC : $bcc"
echo "Subject : $subject"
echo "Body : $body"
echo "Status : $status"
echo "Timestamp : $timesta"
echo $body | mail -s "$subject" $to -c $cc -b $bcc
printf "$msgid,$dat,$to,$cc,$bcc,$subject,$body,Sent,$TIMESTAMP" >> $OUTPUT
done < $INPUT
IFS=$OLDIFS
答案1
外壳只是一个非常糟糕的文本解析工具。您应该只将它用于最简单的任务,甚至可能也不应该。它速度慢、效率低、语法非常复杂,容易出错,而且缺乏最基本的文本处理工具。除此之外,CSV 是一种复杂的格式,允许嵌套和多行条目。尝试在 shell 中解析 CSV 文件只是自找麻烦。
因此,请使用支持 CSV 解析的正确编程语言。例如,在Python中:
#!/bin/python3
import csv
import sys
with open(sys.argv[1], newline='') as csvfile:
fieldnames = ("MsgID","Date","To","CC","BCC",
"Subject","Body","Status","Timestamp")
reader = csv.reader(csvfile, delimiter=',',
for row in reader:
for fieldName, value in zip(fieldnames,row):
print("%s: %s" % (fieldName,value))
我将上面的脚本保存为foo.py
:
MsgID: #1
Date: 13/8/2020
To: [email protected]
CC: [email protected]
BCC:
Subject: Subject
Body: Dear Dude,
Information have been updated. Please login to APP to review by 30th August 2020. Thank you.
Best regards,
Mr. Mack
Status: Pending
Timestamp: 13/8/2020 12:35
如您所见,它已正确识别所有字段。因此,为了也发送邮件,您可以执行以下操作(改编自此处):
#!/bin/python3
import csv
import sys
import subprocess
import os
def send_message(to, cc, bcc, subject, body):
try:
process = subprocess.Popen(['mail', '-s', subject, '-c', cc, '-b', bcc, to],
stdin=subprocess.PIPE)
except Exception as error:
print(error)
process.communicate(body)
with open(sys.argv[1], newline='') as csvfile:
reader = csv.reader(csvfile, delimiter=',')
for row in reader:
(msgID, date, to, cc, bcc, subj, body, status, timestamp) = row
send_message(to, cc, bcc, subj, body)
mail
免责声明:根据您拥有的选项,您似乎正在使用 BSD 。我无法访问安装了该工具的工作系统,因此我无法测试上面的邮件发送脚本。不过,它至少应该是一个好的开始。
答案2
您可以使用 Miller 等工具(https://github.com/johnkerl/miller),专门处理结构化文本数据。
例如从这个input.txt
文件开始
#1,13/8/2020,[email protected],[email protected],,Subject,"Dear Dude,
Information have been updated. Please login to APP to review by 30th August 2020. Thank you.
Best regards,
Mr. Mack",Pending,13/8/2020 12:35
和跑步
mlr --implicit-csv-header --c2x cat then label MsgID,Date,To,CC,BCC,Subject,Body,Status,Timestamp input.txt
你将会拥有
MsgID #1
Date 13/8/2020
To [email protected]
CC [email protected]
BCC
Subject Subject
Body Dear Dude,
Information have been updated. Please login to APP to review by 30th August 2020. Thank you.
Best regards,
Mr. Mack
Status Pending
Timestamp 13/8/2020 12:35
如果你想删除正文字段中的回车符,你可以运行
mlr --implicit-csv-header --c2x cat then label MsgID,Date,To,CC,BCC,Subject,Body,Status,Timestamp then put '$Body=gsub($Body,"\n"," ")' input.txt
具有
MsgID #1
Date 13/8/2020
To [email protected]
CC [email protected]
BCC
Subject Subject
Body Dear Dude, Information have been updated. Please login to APP to review by 30th August 2020. Thank you. Best regards, Mr. Mack
Status Pending
Timestamp 13/8/2020 12:35