优化具有长时间运行的 while 循环的 shell 脚本

Question 1

天啊！

我明白为什么它需要永远运行，你在重复操作，而不是缓存信息，几乎把计算机打死了。可怜的电脑。 :(

awk 不是轻量级的，您要对相同的数据调用它很多很多次。我能够运行一次并设置所有五个变量。

如果不知道这应该做什么或完成什么，那么可以做的事情就太多了。

考虑到所有处理都是 grep、awk、sed 和 tr，通过用 PERL 编写此脚本可以获得令人印象深刻的速度提升。 PERL 被设计用来处理文本和报告。它可以在内部执行所有这些 grep/awk/sed/tr 操作，而无需重复地调用另一个程序。

但这里有一些改进：

if [ -s "$LOC/check.txt" ]; then

function setvars() {
    CONN_TSMP="$1"
    USER="$2"
    HOST="$3"
    DB="$4"
    IP="$5"
    return
}
    while read line; do
        echo " started processing ${line} at $(date) " >> "${SCRIPT_LOC}/running_status.txt"
        ID=$(echo "$line" | tr -d '"')
        # are you sure you don't want the FIRST match?  This will give ALL the matches,
        # which will prevent you from getting good values for the variables
        # to only get first entry that matches:
        # TST=$(grep --max-count=1 -w "$line" "$PERM_LOC/id_processing.txt")
        # (or -m 1, but long options document what you're doing better)
        TST=$(grep -w "$line" "$PERM_LOC/id_processing.txt")
        VARS=$(echo "${TST}" | awk -F '"' '{print "\""$2"\" \""$10"\" \""$18"\" \""$20"\" \""$22'})
        #                                        CONN_TSMP     USER      HOST      IP        DB
        # magic!  setvars receives the 5 values awk pulled out (ran it once!)
        # NO QUOTES on next line, already has them embedded from awk
        setvars $VARS

        if [ -z "$IP" ]; then
            IP="$HOST"
        fi

        CMD_TARGET="/data1/commands/user_commands"
        FILE="${ID}-${CONN_TSMP}-${USER}@${IP}.txt"

        if [ "$USER" == "root" ] && [ -z "$DB" ]; then
            TARGET="/data1/sessions/root_sec"
        else
            TARGET="/data1/sessions/user_sec"
        fi

        # does this need to be redirected to a file?
        ls "$TARGET/$FILE"
        if [ $? -ne 0 ]; then
            # awk can likely do the print and the removal of </> characters in
            # one pass (my awk-fu is weak this morning)
            echo "$TST" | awk -F 'STATUS="0"' '{print $2}'| sed "s/[</>]//g" >> "$TARGET/$FILE"
            echo -e "\n" >> "$TARGET/$FILE"
        fi

        # ALWAYS quote your values, embedded spaces will bite you!
        grep "$line" "$LOC/out.txt" > "$LOC/temp.txt"

        while read val; do
            TSMP=$(echo "$val" | awk -F '"' '{print $2}')
            QUERY=$(echo "$val" | awk -F 'SQLTEXT=' '{print $2}' | sed "s/[\"/]//g")
            echo " TIMESTAMP=$TSMP " >> "$TARGET/$FILE"
            echo " QUERY=$QUERY " >> "$TARGET/$FILE"
            TEXT=$(grep "$QUERY" "$PERM_LOC/commands.txt")
            if [ -n "$TEXT" ]; then
                NUM=$(expr $NUM + 1)
                # could also be:  NUM=$(($NUM+1)) (bash v4.0+)
                SUB_FILE="$ID-$command-$NUM-$TSMP-$USER@$IP.txt"
                echo -e "===============\n" > "$CMD_TARGET/$SUB_FILE"
                echo "FILE      =   \"$SUB_FILE\"" >> "$CMD_TARGET/$SUB_FILE"
                ### same way append 6 more lines to $SUB_FILE

                SUB=$(echo "$WARN_ME" | grep "$command")
                if [ "$command" == "$VC" ]; then
                    STATE=" very critical "
                elif [ -z "$SUB" ]; then
                    STATE=" CRITICAL "
                else
                    STATE=" WARNING "
                fi

                if [ "$USER" != "root" -a "$command" != "$VC" ]; then
                    # this should probably be $command instead of command?
                    # oh wait, probably a placeholder statement
                    mail command &
                elif [ "$USER" == "root" -a -z "$HOST" ]; then
                    mail command &
                elif [ "$USER" == "root" -a "$command" == "$VC" ]; then
                    mail command &
                else
                    echo -e "some message \n" >> $LOC/operations.txt
                fi
            fi
        done < "$LOC/temp.txt"
    done < "$LOC/check.txt"
fi

嗯，“仅限 shell 脚本”。好吧，考虑到这一点，也许您可以预先 grep“$LOC/check.txt”和/或“$LOC/temp.txt”，以便您可以使用“已 grep”输出，而不是在循环中进行 grep。

我看得越多，我就越确信 awk 可以在一次遍历数据的情况下完成所有这些工作...并处理每个条目，而不仅仅是第一个条目（正如我在评论中指出的那样，你确实需要在“read line”和“read var”循环之间进行另一个循环。）

这将是一个很长的 awk 脚本，但绝对可行。而且 awk 值得了解，花点时间玩一下它，它没那么难，只是不同而已。哎呀哎呀！

Answer

天啊！

我明白为什么它需要永远运行，你在重复操作，而不是缓存信息，几乎把计算机打死了。可怜的电脑。 :(