为了合规,我必须保存所有 kafka 原始文档并保留一年。为此,我使用以下内容:
kafka-console-consumer.sh --bootstrap-server kafka1:9092,kafka2:9092,kafka3:9092 --topic test1 --consumer.config /usr/local/kafka_2.12-2.2.1/config/consumer.properties >> /data/backup.txt
已保存的文档具有如下转义字符(每个双引号前面都有 \):
{"type":"Fortigate","@timestamp":"2021-09-06T09:20:38.909Z","message":"<189>date=2021-09-06 time=11:20:38 devname=\"FW_N\" devid=\"FG1123\" logid=\"0000000013\" type=\"traffic\" subtype=\"forward\" level=\"notice\" vd=\"FW-D\" eventtime=1630920038 srcip=1.2.3.4 srcport=59349 srcintf=\"LAG11.2418\" srcintfrole=\"undefined\" dstip=1.2.3.4 dstport=8531 dstintf=\"LAG11.1470\" dstintfrole=\"undefined\" poluuid=\"8f80acf4-db3f-51eb-369f-97e43f065438\" sessionid=3122225826 proto=6 action=\"deny\" policyid=1234 policytype=\"policy\" service=\"gTCP/8531\" dstcountry=\"Reserved\" srccountry=\"Reserved\" trandisp=\"noop\" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat=\"unscanned\" crscore=30 craction=131072 crlevel=\"high\"","@version":"1","host":"5.6.7.8"}
预期结果 :
{"type":"Fortigate","@timestamp":"2021-09-06T09:20:38.909Z","message":"<189>date=2021-09-06 time=11:20:38 devname="FW_N" devid="FG1123" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="FW-D" eventtime=1630920038 srcip=1.2.3.4 srcport=59349 srcintf="LAG11.2418" srcintfrole="undefined" dstip=1.2.3.4 dstport=8531 dstintf="LAG11.1470" dstintfrole="undefined" poluuid="8f80acf4-db3f-51eb-369f-97e43f065438" sessionid=3122225826 proto=6 action="deny" policyid=1234 policytype="policy" service="gTCP/8531" dstcountry="Reserved" srccountry="Reserved" trandisp="noop" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat="unscanned" crscore=30 craction=131072 crlevel="high"","@version":"1","host":"5.6.7.8"}
有什么办法可以告诉 kafka 控制台消费者不要添加这些转义字符?