☰

细节：

tag-icon • tag-icon linux rsyslog json elasticsearch
细节：

在我们公司，我们需要汇总服务统计数据

由于某种原因，我们决定使用 rsyslogd 将应用程序统计信息（json 格式）直接发送到 elasticsearch 系统（本教程）。

但我们遇到了这个问题：

当通过命令将json统计信息发送到本地rsyslogd时logger，一切正常。 在此输入图像描述

但

当应用程序（java）将这些统计信息发送到rsyslogd时（使用积压框架） json 通过反斜杠转义

mmjson解析错误：

9448.407432204:main Q:Reg/w0  : Called action, logging to mmjsonparse
9448.407443744:main Q:Reg/w0  : Action 1 transitioned to state: itx
9448.407450424:main Q:Reg/w0  : entering actionCalldoAction(), state: itx, actionNbr 1
9448.407465385:main Q:Reg/w0  : mmjsonparse: no JSON cookie: '{"subject":                "Report","report_no":              2411,"report_time_from":       1479309445405,"report_time_until":      1479309448406,"report_time_duration":   3001,"upload_lessThan1M_count":      0,"upload_lessThan1M_size":       0,"upload_btw1Mto2M_count":      0,"upload_btw1Mto2M_size":       0,"upload_btw2Mto5M_count":      0,"upload_btw2Mto5M_size":       0,"upload_btw5Mto10M_count":      0,"upload_btw5Mto10M_size":       0,"upload_btw10Mto20M_count":      0,"upload_btw10Mto20M_size":       0,"upload_btw20Mto50M_count":      0,"upload_btw20Mto50M_size":       0,"upload_btw50Mto100M_count":      0,"upload_btw50Mto100M_size":       0,"upload_moreThan100M_count":      0,"upload_moreThan100M_size":       0,"upload_total_size":  0,"upload_total_count": 0,"thumb_count":  0,"thumb_time":   0,"download_lessThan1M_count":      0,"download_lessThan1M_size":       0,"download_btw1Mto2M_count":      0,"download_btw1Mto2M_size":       0,"download_btw2Mto5M_count":      0,"download_btw2Mto5M_size":       0,"download_btw5Mto10M_count":      0,"download_btw5Mto10M_size":       0,"download_btw10Mto20M_count":      0,"download_btw10Mto20M_size":       0,"download_btw20Mto50M_count":      0,"download_btw20Mto50M_size":       0,"download_btw50Mto100M_count":      0,"download_btw50Mto100M_size":       0,"download_moreThan100M_count":      0,"download_moreThan100M_size":       0,"download_total_size":  0,"download_total_count": 0,"cache_served_count": 0,"cache_served_size":  0,"cache_new_count":    0,"cache_new_size":     0}'
9448.407510073:main Q:Reg/w0  : Action 1 transitioned to state: rdy
9448.407517838:main Q:Reg/w0  :     PRIFILT 'local2.*'
9448.407527643:main Q:Reg/w0  :     pmask:  X  X  X  X  X  X  X  X  X  X  X  X  X  X  X  X  X  X FF  X  X  X  X  X  X  X 
9448.407598847:main Q:Reg/w0  : PRIFILT condition result is 1
9448.407604631:main Q:Reg/w0  :     ACTION 2 [omelasticsearch:action(type="omelasticsearch" ...)]
9448.407617909:main Q:Reg/w0  : executing action 2
9448.407622728:main Q:Reg/w0  : Called action, logging to omelasticsearch
9448.407631995:main Q:Reg/w0  : action 3 queue: qqueueAdd: entry added, size now log 1, phys 1 entries

软件详细信息：（rsyslog 8.4.2-1+deb8u2、rsyslog-elasticsearch）

系统日志配置：

#load needed modules
#load needed modules
module(load="imuxsock") # provides support for local system logging
module(load="imklog") # provides kernel logging support
module(load="mmjsonparse") #for parsing CEE-enhanced syslog messages
module(load="omelasticsearch") #for indexing to Elasticsearch


#try to parse structured logs
local2.* action(type="mmjsonparse")

#define a template to print field "foo"
template(name="justFoo" type="list") {
    property(name="$!all-json")
}

#and now let's write the contents of field "foo" in a file
#action(type="omfile"
#    template="justFoo"
#    file="/tmp/foo")

local2.*    action(type="omelasticsearch"
       server="192.168.218.42"
       serverport="9200"
       template="justFoo"
       searchIndex="stats"
       searchType="stats"
       bulkmode="on"
       queue.type="linkedlist"
       queue.size="5000"
       queue.dequeuebatchsize="300"
       action.resumeretrycount="-1")

答案1

我已经找到答案了！

local2.* action(type="mmjsonparse")

将更改为：

action(type="mmjsonparse" cookie="")

细节：

特定于操作的配置指令：

cookie [字符串] 默认为“@cee:”

允许设置必须出现在消息的 JSON 部分前面的 cookie。

最重要的是，可以将其设置为空字符串（“”），以便不需要任何 cookie。在这种情况下，JSON 前面允许有前导空格。 JSON 后面不允许有非空白字符。如果需要，则必须使用 mmnormalize。

参考：http://www.rsyslog.com/doc/v8-stable/configuration/modules/mmjsonparse.html

最重要的是，可以将其设置为空字符串（“”），以便不需要任何 cookie。在这种情况下，JSON 前面允许有前导空格。 JSON 后面不允许有非空白字符。如果需要，则必须使用 mmnormalize。

相关内容