我有一个规则文件。每行都有一个 sid:number 和 rev:number。
我想将此文件与更新的文件进行比较,但是并非所有行都会更新。
如果具有相同 sid:number 的线路之一具有更高的 rev:number,我需要将其替换为更高的 rev:number。
这就是我目前所处的情况
grep -oP "sid:[0-9]{0,11}; rev:[0-9]{0,3}" all_rules.rules |
while read line; do
if grep -q "$line" /home/path/update_rules.rules; then
echo updated;
else
echo > /dev/null;
fi
done
这是 all.rules 文件的示例:
alert udp $HOME_NET any -> any 53 (msg:"ET
TROJAN CopyKittens? Matryoshka DNS Lookup 1 (winupdate64 . com)";
content:"|01 00 00 01 00 00 00 00 00 00|"; depth:10; offset:2;
content:"|0b|winupdate64|03|com|00|"; nocase; distance:0; fast_pattern;
reference:url,www.clearskysec.com/wp-
content/uploads/2017/07/Operation_Wilted_Tulip.pdf; classtype:trojan-
activity; sid:2024495; rev:1;)
alert udp $HOME_NET any -> any 53 (msg:"ET
TROJAN CopyKittens? Matryoshka DNS Lookup 2 (twiter-statics . info)";
content:"|01 00 00 01 00 00 00 00 00 00|"; depth:10; offset:2;
content:"|0e|twiter|2d|statics|04|info|00|"; nocase; distance:0;
fast_pattern; reference:url,www.clearskysec.com/wp-
content/uploads/2017/07/Operation_Wilted_Tulip.pdf;
reference:md5,752240cddda5acb5e8d026cef82e2b54; classtype:trojan-
activity; sid:2024496; rev:1;)
alert udp $HOME_NET any -> any 53 (msg:"ET TROJAN CopyKittens? Cobalt
Strike DNS Lookup (cloudflare-analyse . com)"; content:"|01 00 00 01 00
00 00 00 00 00|"; depth:10; offset:2;
content:"|12|cloudflare|2d|analyse|03|com|00|"; nocase; distance:0;
fast_pattern; threshold:type limit, track by_src, count 1, seconds 60;
reference:url,www.clearskysec.com/wp-
content/uploads/2017/07/Operation_Wilted_Tulip.pdf;
reference:md5,752240cddda5acb5e8d026cef82e2b54; classtype:trojan-
activity; sid:2024497; rev:1;)
这是 update.rules 的示例:
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET TROJAN Revcode
RAT CnC"; flow:established,to_server; content:"POST"; http_method;
content:".php"; http_uri; content:"keyauth="; http_client_body;
fast_pattern; depth:8; content:"&key="; http_client_body; distance:0;
content:"&uid="; http_client_body; distance:0; content:!"Referer|3a|";
http_header; content:"WinHttpRequest"; http_header; metadata:
former_category TROJAN; reference:md5,3f652d9bc17a4be3c0e497ea19848344;
classtype:trojan-activity; sid:2024500; rev:1; metadata:affected_product
Windows_XP_Vista_7_8_10_Server_32_64_Bit, attack_target Client_Endpoint,
deployment Perimeter, signature_severity Major, created_at 2017_07_27,
performance_impact Moderate, updated_at 2017_07_27;)
alert udp $HOME_NET any -> any 53 (msg:"ET TROJAN CopyKittens? Matryoshka
DNS Lookup 1 (winupdate64 . com)"; content:"|01 00 00 01 00 00 00 00 00
00|"; depth:10; offset:2; content:"|0b|winupdate64|03|com|00|"; nocase;
distance:0; fast_pattern; reference:url,www.clearskysec.com/wp-
content/uploads/2017/07/Operation_Wilted_Tulip.pdf; classtype:trojan-
activity; sid:2024495; rev:2;)
alert udp $HOME_NET any -> any 53 (msg:"ET TROJAN CopyKittens Matryoshka DNS
Lookup 2 (twiter-statics . info)"; content:"|01 00 00 01 00 00 00 00 00
00|"; depth:10; offset:2; content:"|0e|twiter|2d|statics|04|info|00|";
nocase; distance:0; fast_pattern; metadata: former_category TROJAN;
reference:url,www.clearskysec.com/wp-
content/uploads/2017/07/Operation_Wilted_Tulip.pdf;
reference:md5,752240cddda5acb5e8d026cef82e2b54; classtype:trojan-activity;
sid:2024496; rev:2; metadata:affected_product
Windows_XP_Vista_7_8_10_Server_32_64_Bit, attack_target Client_Endpoint,
deployment Perimeter, signature_severity Major, created_at 2017_07_25,
malware_family Matryoshka, performance_impact Moderate, updated_at
2017_07_25;)
alert udp $HOME_NET any -> any 53 (msg:"ET TROJAN CopyKittens Cobalt Strike
DNS Lookup (cloudflare-analyse . com)"; content:"|01 00 00 01 00 00 00 00 00
00|"; depth:10; offset:2; content:"|12|cloudflare|2d|analyse|03|com|00|";
nocase; distance:0; fast_pattern; threshold:type limit, track by_src, count
1, seconds 60; metadata: former_category TROJAN;
reference:url,www.clearskysec.com/wp-
content/uploads/2017/07/Operation_Wilted_Tulip.pdf;
reference:md5,752240cddda5acb5e8d026cef82e2b54; classtype:trojan-activity;
sid:2024497; rev:2; metadata:affected_product
Windows_XP_Vista_7_8_10_Server_32_64_Bit, attack_target Client_Endpoint,
deployment Perimeter, signature_severity Major, created_at 2017_07_25,
malware_family CobaltStrike, performance_impact Moderate, updated_at
2017_07_26;)
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET TROJAN
Win32/BanloadDownloader.XZY Retrieving Payload"; flow:to_server,established;
content:"GET"; http_method; content:"/sosdoudou_V3/"; http_uri;
fast_pattern; content:"WinHttp.WinHttpRequest"; http_header;
content:!"Accept-"; http_header; content:!"Referer|3a 20|"; http_header;
metadata: former_category TROJAN;
reference:md5,98376de10118892f0773617da137c2be
md5,599ea45f5420f948e0836239eb3ce772; classtype:trojan-activity;
sid:2024499; rev:2; metadata:affected_product
Windows_XP_Vista_7_8_10_Server_32_64_Bit, attack_target Client_Endpoint,
deployment Perimeter, signature_severity Major, created_at 2017_07_26,
malware_family Banload, performance_impact Moderate, updated_at 2017_07_26;)
请注意,有三个相同的规则sid:2024497
,sid:2024496
但是sid:2024495
update.rules 包含更新修订。我想用 update.rules 中规则的较新版本替换 all.rules 文件中的旧版本。
答案1
这在最少的测试中对我有用:
#!/bin/zsh
typeset -A rule sidrev
while read -r line; do
sid=${${line/*sid:/}/;*/}
rev=${${line/*rev:/}/;*/}
if [[ "$rev" -gt "$sidrev[$sid]" ]]; then
sidrev[$sid]="$rev"
rule[$sid]="$line"
fi
done
echo -E ${(F)rule}
该脚本在 stdin 上读取 snort 规则,并在 stdout 上输出其已读取的所有规则的最新版本。
答案2
做出一些假设以获得某种答案。这些假设是:
- 单向更新:update_rules(或输入文件2)是最新内容的参考文件
- 参考文件将仅包含最新版本或至少与要更新的文件(输入文件 1)相同的版本。这意味着,不检查 rev (IF2) 实际上 >= rev IF1 本身。阿赫东!
- 我依赖于 IF1 和 IF2 之间的差异。不确定两者之间的差异有多大(按行数计算),这可能会产生影响
无论如何,blabla 已经够了,到肉的部分
#!/bin/bash
cp "$1" "copy_$1" #backup file we're going to change
#then only extract sid + rev (cut -f cols) + sort
#do that for both file and diff them with RCS format > see output of script for example
#and filter to keep only the sid (prob worth testing without grep to see output)
var=($(diff -n <(egrep -oe "sid:[0-9]{0,11}; rev:[0-9]{0,3}" $1|sort -k2) <(egrep -oe "sid:[0-9]{0,11}; rev:[0-9]{0,3}" $2|sort -k 2) | egrep -oe "sid:[0-9]{0,11};" ))
#Now loop over each SID
for i in ${var[@]}; do
#Extract line number in IF1
oldline=$(grep -n $i $1|cut -f1 -d:)
#Extract replacement line in IF2
newline=$(grep $i $2.txt)
#awk magic see (note sed was a pain, couldn't get it to work :/) >> https://askubuntu.com/questions/434051/how-to-replace-a-string-on-the-5th-line-of-multiple-text-files
awk -v nline="$oldline" -v repl="$newline" '(NR==nline){$0=repl}1;' "copy_$1" > "f.tmp" && mv "f.tmp" "copy_$1"
done
#then highlight diff between copy and original // you'll still need to mv "copy_$1" "$1" for it to be applied
diff -n "copy_$1" "$1"
将所有这些包装在一个脚本中,然后调用它:
./my_script.sh all_rules.rules /home/path/update_rules.rules
答案3
您没有提到perl
,但我认为既然您提到了awk
、sed
和,就可以使用grep
。我选择perl
而不是awk
主要是因为使用多级关联数组(perl
行话中的“Hash-of-Hashes”或“HoH”)比在awk
.
#!/usr/bin/perl
use strict;
# array used to keep track of the order each sid was first seen,
# so that they can be printed out in the same order.
# Necessary because perl hashes are inherently un-ordered.
my @order=();
# hashed array to contain the highest rev seen of each sid.
my %S = ();
# count of the number of files we've read completely so far.
my $filenum=0;
while(<>) {
s/^\s*|\s*$//g; # strip leading and trailing spaces
if (m/^$/) { $filenum++ if eof; next }; # skip empty lines
# extract the sid and the rev
my ($sid, $rev) = $_ =~ (m/^.*; sid:(\d+); rev:(\d+)/) ;
# store or update an anonymous hash containing the rev and the entire
# line in the hash, keyed by the sid.
if (defined($S{$sid})) {
$S{$sid} = { rev => $rev, line => $_ } if ( ($rev > $S{$sid}->{rev}) );
} else {
next if ($filenum); # only store sid if we're still reading the 1st file.
push @order, $sid;
$S{$sid} = { rev => $rev, line => $_ };
};
$filenum++ if eof;
};
# if you want output sorted by the sid, comment the first of the next
# two lines and uncomment the second
for my $sid (@order) {
#for my $sid (sort keys %S) {
print $S{$sid}->{line}, "\n";
};
将其另存为,例如,./apply-update.pl
并将其运行为./apply-update.pl all.rules update.rules > out.rules
您的新示例输入文件的输出将只有三行(在 中看到的 sidall.rules
已由 更新update.rules
,即“rev:2”而不是“rev:1”版本):
alert udp $home_net any -> any 53 (msg:"et trojan copykittens? matryoshka dns lookup 1 (winupdate64 . com)"; content:"|01 00 00 01 00 00 00 00 00 00|"; depth:10; offset:2; content:"|0b|winupdate64|03|com|00|"; nocase; distance:0; fast_pattern; reference:url,www.clearskysec.com/wp- content/uploads/2017/07/operation_wilted_tulip.pdf; classtype:trojan- activity; sid:2024495; rev:2;)
alert udp $home_net any -> any 53 (msg:"et trojan copykittens matryoshka dns lookup 2 (twiter-statics . info)"; content:"|01 00 00 01 00 00 00 00 00 00|"; depth:10; offset:2; content:"|0e|twiter|2d|statics|04|info|00|"; nocase; distance:0; fast_pattern; metadata: former_category trojan; reference:url,www.clearskysec.com/wp- content/uploads/2017/07/operation_wilted_tulip.pdf; reference:md5,752240cddda5acb5e8d026cef82e2b54; classtype:trojan-activity; sid:2024496; rev:2; metadata:affected_product windows_xp_vista_7_8_10_server_32_64_bit, attack_target client_endpoint, deployment perimeter, signature_severity major, created_at 2017_07_25, malware_family matryoshka, performance_impact moderate, updated_at 2017_07_25;)
alert udp $home_net any -> any 53 (msg:"et trojan copykittens cobalt strike dns lookup (cloudflare-analyse . com)"; content:"|01 00 00 01 00 00 00 00 00 00|"; depth:10; offset:2; content:"|12|cloudflare|2d|analyse|03|com|00|"; nocase; distance:0; fast_pattern; threshold:type limit, track by_src, count 1, seconds 60; metadata: former_category trojan; reference:url,www.clearskysec.com/wp- content/uploads/2017/07/operation_wilted_tulip.pdf; reference:md5,752240cddda5acb5e8d026cef82e2b54; classtype:trojan-activity; sid:2024497; rev:2; metadata:affected_product windows_xp_vista_7_8_10_server_32_64_bit, attack_target client_endpoint, deployment perimeter, signature_severity major, created_at 2017_07_25, malware_family cobaltstrike, performance_impact moderate, updated_at 2017_07_26;)
这会是许多比任何基于 的版本快数倍bash
- shell 解释器的文本处理速度远不及perl
orawk
或python
。像这样的编译语言C
会更快,但是用 C 编写这样的东西可能至少需要 50 或 60 行,甚至可能几百行 C 代码,而不是 15 行 perl(不包括注释或空行) )。