☰

根据第 1 列中的日期查找列中唯一值的计数

tag-icon • tag-icon text-processing awk csv
根据第 1 列中的日期查找列中唯一值的计数

我有 3 个逗号分隔的引用字段。

last crawled,linking page,domain
"Nov 17, 2018","https://allestoringen.be/problemen/bwin/antwerpen","allestoringen.be"
"Aug 11, 2017","http://casino.linkplek.be/","linkplek.be"
"Nov 17, 2018","http://pronoroll.blogspot.com/p/blog-page_26.html","pronoroll.blogspot.com"
etc

我需要删除日期字段上的重复项，并查找每个唯一日期（第 $2 列）的唯一链接页面的数量以及该唯一日期（第 $3 列）的唯一域的数量。我努力了 ：

awk '{A[$1 OFS $2]++} END {for(k in A) print k, A[k]}' FPAT='([^,]*)|("[^"]+")' file
awk '{A[$1 OFS $3]++} END {for(k in A) print k, A[k]}' FPAT='([^,]*)|("[^"]+")' file

但我对一次性获得所有 3 列感到有点困惑。

答案1

# script.awk

BEGIN {
    FPAT = "[^\"]+"
}

{
    # the first if is to avoid processing title row and rows that do not contain fields
    if (NF > 1) {
        # the third field is the linking page column; the second field is a comma
        if ($3 != "" && $1 $3 in unique_linking_pages == 0) {
            unique_linking_pages[$1 $3]
            unique_linking_page_counts[$1]++
        }
        # the fifth field is the domain column; the fourth field is a comma
        if ($5 != "" && $1 $5 in unique_domains == 0) {
            unique_domains[$1 $5]
            unique_domain_counts[$1]++
        }

        # in case of empty fields in columns 2 and or 3 of the input file,
        # this ensures that all the dates are recorded
        dates[$1]
    }
}

END {
    printf "Date, Unique Linking Page Count, Unique Domain Count\n"
    for (date in dates)
        output = output date " | " unique_linking_page_counts[date] " | " unique_domain_counts[date] "\n"

    system("column -t -s '|' <<< '" output "'")
}

相关内容