☰

从自己的日志文件创建 CSV 和统计图表

tag-icon • tag-icon linux logging csv bash-scripting
从自己的日志文件创建 CSV 和统计图表

我只是没有那么擅长让它工作......

我编写了一个小脚本，每 15 分钟运行一次速度测试，并将输出记录在每日日志文件中：

#!/bin/bash

DATE=$(date +%d.%m.%Y_%H:%M)
DAY=$(date +%d.%m.%Y)
WANIP=$(dig +short myip.opendns.com @resolver1.opendns.com)
echo "Timestamp: "$DATE >> /home/rene/speedtests/$DAY.log
echo "WanIP: "$WANIP >> /home/rene/speedtests/$DAY.log

/home/rene/speedtest-cli --server 3199 --share --simple >> /home/rene/speedtests/$DAY.log
echo "----------" >> /home/rene/speedtests/$DAY.log

日志如下：

[rene@manjaro-home ~]$ cat speedtests/11.10.2016.log 
Timestamp: 11.10.2016_00:00
WanIP: 77.119.x.x
Ping: 63.773 ms
Download: 32.90 Mbit/s
Upload: 29.05 Mbit/s
Share results: http://www.speedtest.net/result/5703959727.png
----------
Timestamp: 11.10.2016_00:15
WanIP: 77.119.x.x
Ping: 70.06 ms
Download: 30.85 Mbit/s
Upload: 26.31 Mbit/s
Share results: http://www.speedtest.net/result/5703983637.png
----------
Timestamp: 11.10.2016_00:30
WanIP: 77.119.x.x
Ping: 65.722 ms
Download: 31.16 Mbit/s
Upload: 26.80 Mbit/s
Share results: http://www.speedtest.net/result/5704006209.png
----------

想投诉我的提供商，因为我应该有 150/50，但一个月以来我的带宽一直很差=(

现在我需要一个脚本来转换这个日志文件，如下所示：

Time,Ping,Download,Upload,Link
11.10.2016_00:00,63.773,32.90,29.05,http://www.speedtest.net/result/5703959727.png
11.10.2016_00:15,70.06,30.85,26.31,http://www.speedtest.net/result/5703983637.png
11.10.2016_00:30,65.722,31.16,26.80,http://www.speedtest.net/result/5704006209.png

我已经尝试了一些东西，但我只是想用 awk 来做；-D

[rene@manjaro-home ~]$ cat daily-csv.sh 
#!/bin/bash
DAY=$(date +%d.%m.%Y)

#echo '"Timestamp","Ping","Download","Upload","Link"' > /home/rene/speedtests/$DAY.csv
for line in `cat /home/rene/speedtests/$DAY.log`; do
            timestamp=$(echo $line | awk -F"Timestamp: " '{$0=$1}2');
echo $timestamp
done





#for F in /home/rene/speedtests/$DAY.log
#do
#    {
#        read Timestamp
#        read WanIP
#        read Ping
#        read Download
#        read Upload
#   read Link
#   read Placeholder
#    } < $F
#    echo "$Timestamp,$Ping,$Download,$Upload,$Link" >> speedtests/$DAY.csv
#    
#TIME=${Timestamp#* }
#PING=${Ping#* }
#DOWN=${Download#* }
#UP=${Upload#* }
#LINK=${Link#*: }
#
#echo $TIME,$PING,$DOWN,$UP,$LINK
#
#
#done

注释掉的部分只给出了第一条日志。

那么...有人能帮助我吗？:)

答案1

首先，我正在使用您上传的脚本，但它在我的环境中（Solaris 10）不起作用，所以这是对您的问题的回答：

    #!/bin/bash

    #######################################################
    # Script to convert with SED and AWK a logfile to CSV #
    #######################################################


    ######################
    ##    Empty files   ##
    ######################

    cat /dev/null>a.txt
    cat /dev/null>b.txt
    cat /dev/null>c.txt
    cat /dev/null>d.txt
    cat /dev/null>e.txt

    cat /dev/null>one.txt
    cat /dev/null>two.txt
    cat /dev/null>three.txt
    cat /dev/null>four.txt
    cat /dev/null>five.csv


    #########################################
    ##    Separate all aoutputs in files   ##
    #########################################


    for line in `cat speedtests/11.10.2016.log | grep "Timestamp: "`;
    do
    echo $line |awk -F"Timestamp: " '{print $1}' | sed '/^$/d';
    done>a.txt
    cat a.txt;



    for line in `cat speedtests/11.10.2016.log | grep "Ping"`;
    do
    echo $line |awk -F"Ping:" '{print $1}' | sed '/ms/d' |sed '/^$/d';
    done>b.txt;
    cat b.txt;



    for line in `cat speedtests/11.10.2016.log | grep "Download: "`;
    do
    echo $line |awk -F"Download: " '{print $1}' | sed '/Mbit/d' | sed '/^$/d';
    done>c.txt
    cat c.txt


    for line in `cat speedtests/11.10.2016.log | grep "Upload: "`;
    do
    echo $line |awk -F"Upload: " '{print $1}' | sed '/Mbit/d' | sed '/^$/d';
    done>d.txt
    cat d.txt


    for line in `cat speedtests/11.10.2016.log | grep "http"`;
    do
    echo $line |awk -F":" '{print $2}'  |sed '/^$/d' | sed  '1s/./https:\//' | sed 's/\//\\\\\//g' 
    done>e.txt
    cat e.txt


    #################################################################
    ##    Create outputs from previous outputs and adding a coma   ##
    #################################################################

    while read var
    do
    sed "s/$/,$var/" a.txt > one.txt
    done<b.txt
    cat one.txt;

    while read var
    do
    sed "s/$/,$var/" one.txt > two.txt
    done<c.txt
    cat two.txt;

    while read var
    do
    sed "s/$/,$var/" two.txt > three.txt
    done<d.txt
    cat three.txt;


    while read var
    do
    sed "s/$/,$var/" three.txt > four.txt
    done<e.txt
    cat four.txt;

    ##########################################
    ##    Add header only if you requided   ##
    ##########################################

    echo "------Time---------Ping--Down---Up--------Link-------------------------------------" >five.csv

    cat four.txt >>five.csv

    ###############################################
    ##    This is the final output as you whish  ##
    ###############################################

    cat five.csv;

请反馈

问候路易斯

相关内容