![Fluent-Bit - 如何提取文件名的一部分并将其附加到 Cloudwatch 日志组的 log_stream_name](https://linux22.com/image/794199/Fluent-Bit%20-%20%E5%A6%82%E4%BD%95%E6%8F%90%E5%8F%96%E6%96%87%E4%BB%B6%E5%90%8D%E7%9A%84%E4%B8%80%E9%83%A8%E5%88%86%E5%B9%B6%E5%B0%86%E5%85%B6%E9%99%84%E5%8A%A0%E5%88%B0%20Cloudwatch%20%E6%97%A5%E5%BF%97%E7%BB%84%E7%9A%84%20log_stream_name.png)
刚接触 Fluent-Bit 并寻求一些相关指导。
最新版本的 Fluent-Bit 已安装在 ec2 上,并成功运行。我正在将日志从 /var/log/nginx... 推送到 Cloudwatch
例如,我在 /var/log/nginx/client.test-access.log 中有日志
我想使用 Fluent-Bit 提取“client.test”并将其附加到 log_stream_name,以便它在 Cloudwatch 日志流中显示为 client.test
我一直在尝试使用 rewrite_tag
[INPUT]
Name tail
Path /var/log/nginx/*access.log
Tag nginx.access
Path_Key filepath
[FILTER]
Name rewrite_tag
Match nginx.access
# Example assumes log file name is e.g. *.test.com-access.log
# with a destination log group of /aws/ec2/access-logs
Rule $filepath ^/var/log/nginx/(.*)-access.log$ nginx.access.$1
Emitter_Name re_emitted
[OUTPUT]
Name cloudwatch_logs
Match nginx.access.*
region eu-west-2
log_group_name /aws/ec2/access-logs
log_stream_name fluent-bit-${TAG[2]}
auto_create_group true
但它在云监控日志流中显示为空白,有人可以建议我这样做是否正确或者是否有更好的方法吗?
谢谢
答案1
您可以尝试使用记录访问器结合Lua添加domain
从中提取的字段filepath
。然后您可以附加domain
到log_stream_name
。
从您的配置中:
[INPUT]
Name tail
Path /var/log/nginx/*access.log
Tag nginx.access
Path_Key filepath
[FILTER]
Name lua
Match nginx.access
call add_domain_field
script domain.lua
[OUTPUT]
Name cloudwatch_logs
Match nginx.access
region eu-west-2
log_group_name /aws/ec2/access-logs
log_stream_name fluent-bit
log_stream_template fluent-bit-$domain
auto_create_group true
并且在domain.lua
。此文件应位于您的配置的同一目录中fluentbit
:
function add_domain_field(tag, timestamp, record)
local filepath = record["filepath"]
-- https://www.lua.org/manual/5.4/manual.html#pdf-string.match
local domain = string.match(filepath, "^/var/log/nginx/(.*)-access.log$")
record["domain"] = domain
-- code 2 means the timestamp is not changed, only the record is changed
-- https://docs.fluentbit.io/manual/pipeline/filters/lua#return-values
return 2, timestamp, record
end