![如何将文件的内容写入新文件,删除重复的行](https://linux22.com/image/82025/%E5%A6%82%E4%BD%95%E5%B0%86%E6%96%87%E4%BB%B6%E7%9A%84%E5%86%85%E5%AE%B9%E5%86%99%E5%85%A5%E6%96%B0%E6%96%87%E4%BB%B6%EF%BC%8C%E5%88%A0%E9%99%A4%E9%87%8D%E5%A4%8D%E7%9A%84%E8%A1%8C.png)
例如,一个文件file1.txt
包含
Hi how are you
hello
today is monday
hello
I am fine
Hi how are you
处理后file1.txt
应该写入file2.txt
,内容应该像这样,而不重复相同的行。
Hi how are you
hello
today is monday
I am fine
我可以使用什么命令在 Linux 终端中执行此操作?
答案1
start cmd:> awk 'lines[$0]++ == 0' input
Hi how are you
hello
today is monday
I am fine
答案2
这是一项简单的工作sort
,使用以下唯一的 ( -u
) 选项sort
:
% sort -u file1.txt
hello
Hi how are you
I am fine
today is monday
要将其保存在file2.txt
:
sort -u file1.txt >file2.txt
如果您想保留初始顺序:
% nl file1.txt | sort -uk2,2 | sort -k1,1n | cut -f2
Hi how are you
hello
today is monday
I am fine