☰

Logstash - 根据包含的数组中的对象将事件分成两部分

tag-icon • tag-icon logstash kibana
Logstash - 根据包含的数组中的对象将事件分成两部分

这是我第一次使用 logstash，我正在尝试从 amavisd-new 获取 JSON 报告以进行搜索和分析。Amavisd-new 能够将 JSON 日志写入 redis，我已经完美导入了所有内容，并开始学习所有这些方法。

但我有一个问题 - amavis 的 JSON 报告格式如下所示 - 请注意“收件人”有一个数组，每个收件人一个条目。

我想将整个事件分成两部分 - 每个收件人一个，保持其他所有字段不变，但将每个收件人数组成员中的“action”、“ccat_main”、“queued_as”等字段替换到主事件中。

这个想法是，一个有两个接收者的传入事件将导致 logstash 中出现两个单独的日志事件 - 每个人一个。

我已经研究过事件的拆分，但我不知道如何做到这一点——我似乎无法在任何地方找到任何合适的例子。

因此，对于真实的例子，给出如下：

 {
    "@timestamp" => "2014-05-06T09:29:47.048Z",
    "time_unix" => 1399368587.048,
    "time_iso_week_date" => "2014-W19-2",
    "partition" => "19",
    "type" => "amavis",
    "host" => "mailer.example.net",
    "queued_as" => ["3gNFyR4Mfjzc3", "3gNFyR4n6Lzc4"],
    "recipients" => [
      { "action" => "PASS",
        "ccat_main" => "Clean",
        "queued_as" => "3gNFyR4Mfjzc3",
        "rcpt_is_local" => false,
        "rcpt_to" => "[email protected]",
        "smtp_code" => "250",
        "smtp_response" => "250 2.0.0 from MTA(smtp:[::1]:10013): 250 2.0.0 Ok: queued as 3gNFyR4Mfjzc3",
        "spam_score" => -2.0
      },
      { "action" => "PASS",
        "ccat_main" => "Clean",
        "mail_id_related" => "men7HTERZaOF",
        "penpals_age" => 1114599,
        "queued_as" => "3gNFyR4n6Lzc4",
        "rcpt_is_local" => true,
        "rcpt_to" => "[email protected]",
        "smtp_code" => "250",
        "smtp_response" => "250 2.0.0 from MTA(smtp:[::1]:10013): 250 2.0.0 Ok: queued as 3gNFyR4n6Lzc4",
        "spam_score" => -5.272
      }
    ],
    "smtp_code"  => ["250"],
  }

我希望最终有两个不同的事件，如下所示：

  {
    "@timestamp" => "2014-05-06T09:29:47.048Z",
    "time_unix" => 1399368587.048,
    "time_iso_week_date" => "2014-W19-2",
    "partition" => "19",
    "type" => "amavis",
    "host" => "mailer.example.net",
    "queued_as" => ["3gNFyR4Mfjzc3", "3gNFyR4n6Lzc4"],
    "action" => "PASS",
    "ccat_main" => "Clean",
    "queued_as" => "3gNFyR4Mfjzc3",
    "rcpt_is_local" => false,
    "rcpt_to" => "[email protected]",
    "smtp_code" => "250",
    "smtp_response" => "250 2.0.0 from MTA(smtp:[::1]:10013): 250 2.0.0 Ok: queued as 3gNFyR4Mfjzc3",
    "spam_score" => -2.0
    "smtp_code"  => ["250"],
  }

和

  {
    "@timestamp" => "2014-05-06T09:29:47.048Z",
    "time_unix" => 1399368587.048,
    "time_iso_week_date" => "2014-W19-2",
    "partition" => "19",
    "type" => "amavis",
    "host" => "mailer.example.net",
    "queued_as" => ["3gNFyR4Mfjzc3", "3gNFyR4n6Lzc4"],
    "recipients" => [
    "action" => "PASS",
    "ccat_main" => "Clean",
    "mail_id_related" => "men7HTERZaOF",
    "penpals_age" => 1114599,
    "queued_as" => "3gNFyR4n6Lzc4",
    "rcpt_is_local" => true,
    "rcpt_to" => "[email protected]",
    "smtp_code" => "250",
    "smtp_response" => "250 2.0.0 from MTA(smtp:[::1]:10013): 250 2.0.0 Ok: queued as 3gNFyR4n6Lzc4",
    "spam_score" => -5.272
    "smtp_code"  => ["250"],
  }

我怎样才能做到这一点？

谢谢-

汤姆

答案1

另外，很久以前就有人问过，这里有两个链接可以看看： https://www.elastic.co/guide/en/logstash/current/plugins-filters-metricize.html https://www.elastic.co/guide/en/logstash/current/plugins-filters-clone.html

相关内容