Collectd 和 graphite 每 5 分钟导入一次数据，而不是每 1 分钟

2024-5-28 • tag-icon

Collectd 和 graphite 每 5 分钟导入一次数据，而不是每 1 分钟

我对 graphite 还不太熟悉，所以请耐心听我说。我正在寻找大型且相当笨重的 cacti 安装的替代方案，所以我一直在使用 graphite。我们通过 SNMP 提取大量数据，因此我还下载、编译并安装了 collectd，以将 SNMP 数据传输到 graphite。

我在 collectd 中设置了一个简单的查询，只用于获取当前的eth0输入/输出计数器。我希望以一分钟的分辨率捕获一周的数据，然后以 5 分钟的分辨率捕获一周的数据，因此我的storage-schemas.conf数据如下所示：

[carbon]
 pattern = ^carbon\.
 retentions = 60:90d

[default]
 pattern = .*
 retentions = 60s:1w, 5m:1y

同样，collectd.conf我设置了以下内容：

<Plugin snmp>
   <Data "std_traffic">
       Type "if_octets"
       Table true
       Instance "IF-MIB::ifDescr"
       Values "IF-MIB::ifInOctets" "IF-MIB::ifOutOctets"
   </Data>

   <Host "lonsbrndlb01">
       Address "lonsbrndlb01"
       Version 2
       Community "public"
       Collect "std_traffic"
       Interval 60
   </Host>
</Plugin>

这几乎运行完美。按键以石墨形式显示，数据也输入其中。

唯一的问题是数据是计数器，而不是每分钟的速率。我可以使用函数在 graphite 中解决这个问题derivative，该函数据称可以将计数器转换为每分钟的速率。但是，这样做，我看到了以下图表：

很明显，数据每 5 分钟才到达一次，而不是我指定的每 60 秒到达一次。这是为什么？我以为我在 collectd 和 graphite 中都设置了正确的值，所以我想我漏掉了什么。

编辑

关于此内容的更多数据可能会有用。

我使用的公式只是derivative(lonsbrndlb01.snmp.if_octets-eth0.tx)和derivative(lonsbrndlb01.snmp.if_octets-eth0.rx)，尽管由于计数器翻转，我现在已改用非负导数。我还更新了下图以提供比例感。

运行whisper-dump.py该rx.wsp文件会得到以下标题：

Meta data:
  aggregation method: average
  max retention: 31536000
  xFilesFactor: 0.5

Archive 0 info:
  offset: 40
  seconds per point: 60
  points: 10080
  retention: 604800
  size: 120960

Archive 1 info:
  offset: 121000
  seconds per point: 300
  points: 105120
  retention: 31536000
  size: 1261440

后面跟着约2.4M的数据。

附加的图表数据&format=json为：

[{"target": "nonNegativeDerivative(lonsbrndlb01.snmp.if_octets-eth0.rx)", "datapoints": [[null, 1342597800], [26346975.0, 1342597860], [35197821.0, 1342597920], [138121.0, 1342597980], [108605.0, 1342598040], [690712.0, 1342598100], [27213713.0, 1342598160], [876898.0, 1342598220], [463897.0, 1342598280], [137499.0, 1342598340], [96980.0, 1342598400], [26237641.0, 1342598460], [35094898.0, 1342598520], [112569.0, 1342598580], [274897.0, 1342598640], [139174.0, 1342598700], [806881.0, 1342598760], [26206311.0, 1342598820], [112298.0, 1342598880], [781205.0, 1342598940], [606872.0, 1342599000], [5184462.0, 1342599060], [61946135.0, 1342599120], [4126005.0, 1342599180], [115908.0, 1342599240], [714159.0, 1342599300], [195738.0, 1342599360], [26261781.0, 1342599420], [100503.0, 1342599480], [751322.0, 1342599540], [930865.0, 1342599600], [230666.0, 1342599660], [59636.0, 1342599720], [62575579.0, 1342599780], [104950.0, 1342599840], [1208886.0, 1342599900], [379369.0, 1342599960], [785827.0, 1342600020], [26215475.0, 1342600080], [221604.0, 1342600140], [351866.0, 1342600200], [231163.0, 1342600260], [211398.0, 1342600320], [70770807.0, 1342600380], [429324.0, 1342600440], [1937893.0, 1342600500], [1476961.0, 1342600560], [72383.0, 1342600620], [371513.0, 1342600680], [29186024.0, 1342600740], [1924055.0, 1342600800], [280068.0, 1342600860], [341216.0, 1342600920], [36643885.0, 1342600980], [26708952.0, 1342601040], [259828.0, 1342601100], [488406.0, 1342601160], [230698.0, 1342601220], [766407.0, 1342601280], [28252848.0, 1342601340]]}, {"target": "nonNegativeDerivative(lonsbrndlb01.snmp.if_octets-eth0.tx)", "datapoints": [[null, 1342597800], [26007032.0, 1342597860], [34808859.0, 1342597920], [100498.0, 1342597980], [91818.0, 1342598040], [649666.0, 1342598100], [26566941.0, 1342598160], [895897.0, 1342598220], [478867.0, 1342598280], [100242.0, 1342598340], [81130.0, 1342598400], [25908859.0, 1342598460], [34659481.0, 1342598520], [75295.0, 1342598580], [285061.0, 1342598640], [103644.0, 1342598700], [824177.0, 1342598760], [25884962.0, 1342598820], [93420.0, 1342598880], [799160.0, 1342598940], [582373.0, 1342599000], [5024696.0, 1342599060], [61269813.0, 1342599120], [3336907.0, 1342599180], [436657.0, 1342599240], [696692.0, 1342599300], [182144.0, 1342599360], [25947578.0, 1342599420], [79011.0, 1342599480], [733857.0, 1342599540], [1015395.0, 1342599600], [184960.0, 1342599660], [48026.0, 1342599720], [61462810.0, 1342599780], [89187.0, 1342599840], [1195360.0, 1342599900], [386772.0, 1342599960], [744445.0, 1342600020], [25913548.0, 1342600080], [201978.0, 1342600140], [344650.0, 1342600200], [199421.0, 1342600260], [208959.0, 1342600320], [69924581.0, 1342600380], [381593.0, 1342600440], [1610764.0, 1342600500], [1484192.0, 1342600560], [41585.0, 1342600620], [373375.0, 1342600680], [28478208.0, 1342600740], [1893711.0, 1342600800], [253921.0, 1342600860], [354558.0, 1342600920], [36199040.0, 1342600980], [26395675.0, 1342601040], [239238.0, 1342601100], [477775.0, 1342601160], [212554.0, 1342601220], [752374.0, 1342601280], [27890202.0, 1342601340]]}]

它可能是峰值数据，但这个盒子不可能每隔几分钟就达到 60MBit 的流量峰值。

答案1

如果您对相应的 whisper 文件使用 whisper-dump.py 命令，它会显示什么？从图表上看，它似乎不是每 5 分钟一次。您是否有可能只是获得了尖峰网络流量？此外，对于计数器，使用 nonNegativeDerivative 而不是 Derivative 始终是一个好主意，因为 nonNegative 版本考虑了翻转。

答案1

相关内容