指数滚动延迟

指数滚动延迟

我制定了ILM以下政策,

{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_docs": 30
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "15m",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

索引设置如下,

{
  "index": {
    "lifecycle": {
      "name": "index-roll+doc-over",
      "rollover_alias": "router01"
    },
    "number_of_shards": "1",
    "number_of_replicas": "0"
  }
}

正如您已经猜到的那样,我正在测试此 ILM 滚动策略,这就是我仅设置文档大小的原因30。我计划将文档大小限制为 15 亿。但我在创建滚动索引并切换到新索引时观察到巨大的延迟。它正在写入当前索引而不是滚动。对于测试,我仅提取了500日志行。

索引详细信息Index details

health status index                    uuid                   pri rep docs.count docs.deleted store.size pri.store.size

green  open   router01-000001          654G2ArFQiypDn9U69jedw   1   0        500            0    151.1kb        151.1kb

而在撰写此问题时,另一个索引(router01-000001)尚未创建。

以下_ilm/explain是,

{
  "indices" : {
    "router01-000001" : {
      "index" : "router01-000001",
      "managed" : true,
      "policy" : "index-roll+doc-over",
      "lifecycle_date_millis" : 1583819391191,
      "age" : "14.08m",
      "phase" : "hot",
      "phase_time_millis" : 1583819391396,
      "action" : "rollover",
      "action_time_millis" : 1583819706013,
      "step" : "check-rollover-ready",
      "step_time_millis" : 1583819706013,
      "phase_execution" : {
        "policy" : "index-roll+doc-over",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_docs" : 30
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "version" : 1,
        "modified_date_in_millis" : 1583756555777
      }
    }
  }
}

Elasticsearch 当前有2GB堆,logstash 有 1GB。总内存 4GB。

我的问题是,

  • 是否预期有延迟(可能是我误解了翻转概念)
  • 我有哪里弄错了吗?
  • 如果不能,那么我该怎么做才能改善延迟,因为每天将有 2000 万个日志被提取

相关内容