☰

Graylog 流获取事件，但是为空

tag-icon • tag-icon logging elk graylog
Graylog 流获取事件，但是为空

我已经开始将 Palo Alto 日志发送到 Graylog，流规则通过在“标签”字段中匹配“Palo Alto”​​来挑选它们（这就是我的所有流规则；前端 Logstash 实例在发送到 Graylog 之前进行标记）。

我知道 Graylog 节点正在网络接口上接收这些事件： tcpdump 事件

并且流显示它正在获取事件（注意“每秒 22 条消息”）：

注意它目前每秒接收 22 条消息

然而，当我点击 Stream（或搜索 --> 标签：“Palo Alto”​​）时，没有找到任何事件。

空流

我在网上看到的唯一常见问题是使用时区设置将这些事件放到未来，但是我们 Palo Alto Panorama 发送器上的时间是正确的（PST），并且尝试进行未来一天的绝对时间搜索却没有任何结果。

版本信息：

Graylog 2.2.2+691b4b7，代号 Stiegl

Elasticsearch 2.4.4

Lucene 5.5.2

我也有这个问题关于搜索功能无法正常找到实际到达的事件，尚未得到解答。我怀疑这两者之间是否有任何关系，但为了完整性，我将在此包含这一点。

答案1

在 Graylog 服务器节点的 /var/log/graylog-server/server.log 日志文件中，我注意到很多错误，例如：

[54]：索引 [graylog_2]，类型 [message]，id [edb8ec50-1320-11e7-92de-005056b541f6]，消息 [MapperParsingException[无法解析 [ReceiveTime]]；嵌套：IllegalArgumentException[格式无效：“2017/03/27 12:09:40”在“/03/27 12:09:40”处格式错误]；]

所以问题是这些消息可以正常进入 Graylog，但无法被 Elasticsearch 索引。我最终删除并修改了问题字段，直到 Graylog 满意为止。

if "Palo Alto" in [tags] {
    grok {
        match => ["message", "<\d*>(?<patimestamp>\w* \d* \d*:\d*:\d*) (?<PanoramaHost>[^ ]*) (?<FutureUse0>[^,]*),(?<ReceiveTime>[^,]*),(?<SerialNumber>[^,]*),(?<PAType>[^,]*),%{GREEDYDATA:pamessage}"]
    }
    if [PAType] == "SYSTEM" {
        csv {source => "[pamessage]" columns => ["Subtype","FutureUse1","GeneratedTime","vsys","paEventID","Object","FutureUse2","FutureUse3","Module","Severity","Description","SeqNum","ActionFlags"]}
        mutate {remove_field => ["ReceiveTime"] remove_field => ["GeneratedTime"] gsub => ["message", "/", "_"]}
    } else if [PAType] == "TRAFFIC" {
        csv {source => "[pamessage]" columns => ["Threat-ContentType","ConfigVersion","GenerateTime","SrcAddress","DstAddress","NATSrcIP","NATDstIP","Rule","SrcUser","DstUser","App","VSys","SrcZone","DstZone","InboundInterface","OutboundInterface","LogAction","TimeLogged","SessionID","RepeatCount","SrcPort","DstPort","NATSrcPort","NATDstPort","Flags","Protocol","Action","Bytes","BytesSent","BytesReceived","Packets","StartTime","ElapsedTimeInSec","Category","Padding","SeqNum","ActionFlags","SrcCountry","DstCountry","cpadding","pkts_sent","pkts_received"]}
                    mutate {remove_field => ["ReceiveTime"] remove_field => ["GeneratedTime"] gsub => ["message", "/", "_"]}
    } else if [PAType] == "THREAT" {
        csv {source => "[pamessage]" columns => ["Subtype","FutureUse1","GeneratedTime","SrcIP","DstIP","NATSrcIP","NATDstIP","Rule","SrcUser","DstUser","App","vsys","SrcZone","DstZone","IngressInterface","EgressInterface","LogFwdProfile","FutureUse2","SessionID","RepeatCount","SrcPort","DstPort","NATSrcPort","NATDstPort","Flags","Protocol","Action","Misc","ThreatID","Category","Severity","Direction","SeqNum","ActionFlags","SrcLocation","DstLocation","FutureUse3","ContentType","pcapID","Filedigest","Cloud","FutureUse4","UserAgent","FileType","XForwardedFor","Referer","Sender","Subject","Recipient","ReportID"]}
                    mutate {remove_field => ["ReceiveTime"] remove_field => ["GeneratedTime"] gsub => ["message", "/", "_"]}
    } else if [PAType] == "CONFIG" {
        csv {source => "[pamessage]" columns => ["Subtype","FutureUse1","GeneratedTime","Host","vsys","Command","Admin","Client","Result","ConfigPath","SeqNum","ActionFlags","BeforeChangeDetail","AfterChangeDetail"]}
                    mutate {remove_field => ["ReceiveTime"] remove_field => ["GeneratedTime"] gsub => ["message", "/", "_"]}
    } else if [PAType] == "HIP-MATCH" {
        csv {source => "[pamessage]" columns => ["Subtype","FutureUse1","GeneratedTime","SrcUser","vsys","MachineName","OS","SrcAddress","HIPType","FutureUse2","FutureUse3","SeqNum","ActionFlags"]}
                    mutate {remove_field => ["ReceiveTime"] remove_field => ["GeneratedTime"] gsub => ["message", "/", "_"]}
    } else {
        mutate {add_tag => "Uncategorized"}
    }
}

相关内容