我正在使用 Ubuntu 服务器，其中安装了从服务器设置（snap）安装的 Docker 服务，并且 Prometheus 在 Docker 容器中运行。
服务器：
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy
Docker
sudo snap services
Service Startup Current Notes
docker.dockerd enabled active -
docker.nvidia-container-toolkit enabled inactive -
lxd.activate enabled inactive -
lxd.daemon enabled inactive socket-activated
lxd.user-daemon enabled inactive socket-activated
折断
snap 2.61.1
snapd 2.61.1
series 16
ubuntu 22.04
kernel 5.15.0-91-generic
Prometheus docker-compose.yml
version: '3.8'
volumes:
prometheus_data: {}
services:
prometheus:
image: prom/prometheus:latest
container_name: prometheus
restart: unless-stopped
volumes:
- /home/xxxx/data/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "9090:9090/tcp"
Docker IP：
ip addr show docker0
8: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:18:5c:09:7f brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:18ff:fe5c:97f/64 scope link
valid_lft forever preferred_lft forever
问题
因此，我想要做的是获取在 Docker 容器中运行的 Prometheus 服务器的指标。我正在使用以下文档：https://docs.docker.com/config/daemon/prometheus/
一开始就这么说：添加以下配置：
Linux: /etc/docker/daemon.json
{
"metrics-addr": "127.0.0.1:9323"
}
我也尝试过：
{
"metrics-addr": "172.17.0.1:9323"
}
经过一番麻烦之后，我在 askubuntu 上找到了以下问题/答案：https://askubuntu.com/questions/1296892/snap-edit-docker-configuration
所以我将 daemon.json 添加到
/var/snap/docker/current/etc/docker
并使用以下命令重新启动服务：
sudo snap restart docker
重启后，我仍然没有从主机获取任何指标。我使用以下命令进行测试：
curl http://localhost:9323/metrics
curl: (7) Failed to connect to localhost port 9323 after 0 ms: Connection refused
curl http://172.17.0.1:9323/metrics
curl: (7) Failed to connect to 172.17.0.1 port 9323 after 0 ms: Connection refused
Docker 系统信息
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., v0.10.4)
compose: Docker Compose (Docker Inc., v2.17.2)
Server:
Containers: 20
Running: 20
Paused: 0
Stopped: 0
Images: 20
Server Version: 20.10.24
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38
runc version:
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
cgroupns
Kernel Version: 5.15.0-91-generic
Operating System: Ubuntu Core 22
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 31.26GiB
Name: blackholeport
ID: AMKQ:XQSW:OPFY:ZLWE:VCVI:BKBB:NVO4:5WYJ:W6AZ:JB7M:7T4S:WXNC
Docker Root Dir: /var/snap/docker/common/var-lib-docker
Debug Mode: false
Username: hagbart
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
docker 检查 prometheus
[
{
"Id": "ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684",
"Created": "2024-01-19T10:23:03.520072505Z",
"Path": "/bin/prometheus",
"Args": [
"--config.file=/etc/prometheus/prometheus.yml"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 83237,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-01-19T13:00:24.320370241Z",
"FinishedAt": "2024-01-19T13:00:17.400204943Z"
},
"Image": "sha256:2a72b385beafd2b3685ad337d14d23f6851f0ff335c5cd9a04c692b3eb846196",
"ResolvConfPath": "/var/snap/docker/common/var-lib-docker/containers/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684/resolv.conf",
"HostnamePath": "/var/snap/docker/common/var-lib-docker/containers/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684/hostname",
"HostsPath": "/var/snap/docker/common/var-lib-docker/containers/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684/hosts",
"LogPath": "/var/snap/docker/common/var-lib-docker/containers/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684-json.log",
"Name": "/prometheus",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "docker-default",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/home/lukas/data/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:rw"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "prometheus_default",
"PortBindings": {
"9090/tcp": [
{
"HostIp": "",
"HostPort": "9090"
}
]
},
"RestartPolicy": {
"Name": "unless-stopped",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": [
"host.docker.internal:host-gateway"
],
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"Mounts": [
{
"Type": "volume",
"Source": "prometheus_prometheus_data",
"Target": "/prometheus",
"VolumeOptions": {}
}
],
"MaskedPaths": [
"/proc/asound",
"/proc/acpi",
"/proc/kcore",
"/proc/keys",
"/proc/latency_stats",
"/proc/timer_list",
"/proc/timer_stats",
"/proc/sched_debug",
"/proc/scsi",
"/sys/firmware"
],
"ReadonlyPaths": [
"/proc/bus",
"/proc/fs",
"/proc/irq",
"/proc/sys",
"/proc/sysrq-trigger"
]
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/snap/docker/common/var-lib-docker/overlay2/5656a88cbb1d0e0fd4ed939b0cdb30de86b7d2e4af2afd06d2ea33d8dd110199-init/diff:/var/snap/docker/common/var-lib-docker/overlay2/0aa7d62082adf6dcde04943d937cac5dae5d0609b12ae7319421cbf63131fca3/diff:/var/snap/docker/common/var-lib-docker/overlay2/59878a50f72e66a191110dadd15bf94d352cb1e0a543cf6ab0e96f210d9a16e5/diff:/var/snap/docker/common/var-lib-docker/overlay2/d7dd0384e6e888b48e94d3f32bf1e504c2ed1d11526ba5caba7a6cbaa43f1304/diff:/var/snap/docker/common/var-lib-docker/overlay2/8943641d04172c565f42e75963910fdf6df436bc60d3929284c657b16d831dd1/diff:/var/snap/docker/common/var-lib-docker/overlay2/6c943a9276d504534ec9c66ade625447ad516058dce64053b899283aac500395/diff:/var/snap/docker/common/var-lib-docker/overlay2/416d42bd54e76f8a2c78da2799ad6e6c5d2c3d78b822ed8e17ada9bdc0fc17a7/diff:/var/snap/docker/common/var-lib-docker/overlay2/89ab8fa214a8141179afa2c21071e565dc841d1e0f5a69dc372d5b6514456cb0/diff:/var/snap/docker/common/var-lib-docker/overlay2/d99204b78e6ddf1279d8f1b099318f921a61e9017d457955e5a53c5c2d17be8a/diff:/var/snap/docker/common/var-lib-docker/overlay2/259c1d3c5a275a0c51fe499c1a8a2857427cf592eb094dea65447b591ec66058/diff:/var/snap/docker/common/var-lib-docker/overlay2/721958698d3add899bd2597d3dab22267c3aaa40dc556eb7f0cdb8c564b14a74/diff:/var/snap/docker/common/var-lib-docker/overlay2/c122e772ebb24d66d663577b099800060d10e1d1fda0b38e75ea73bc2fdf6273/diff:/var/snap/docker/common/var-lib-docker/overlay2/b1a9df93abc1fbba6b8f3644eaf5ec9a76b8b2193ad1d934241242fb54b74cd0/diff",
"MergedDir": "/var/snap/docker/common/var-lib-docker/overlay2/5656a88cbb1d0e0fd4ed939b0cdb30de86b7d2e4af2afd06d2ea33d8dd110199/merged",
"UpperDir": "/var/snap/docker/common/var-lib-docker/overlay2/5656a88cbb1d0e0fd4ed939b0cdb30de86b7d2e4af2afd06d2ea33d8dd110199/diff",
"WorkDir": "/var/snap/docker/common/var-lib-docker/overlay2/5656a88cbb1d0e0fd4ed939b0cdb30de86b7d2e4af2afd06d2ea33d8dd110199/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/home/lukas/data/config/prometheus/prometheus.yml",
"Destination": "/etc/prometheus/prometheus.yml",
"Mode": "rw",
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "prometheus_prometheus_data",
"Source": "/var/snap/docker/common/var-lib-docker/volumes/prometheus_prometheus_data/_data",
"Destination": "/prometheus",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "ded87ebe4daf",
"Domainname": "",
"User": "nobody",
"AttachStdin": false,
"AttachStdout": true,
"AttachStderr": true,
"ExposedPorts": {
"9090/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"--config.file=/etc/prometheus/prometheus.yml"
],
"Image": "prom/prometheus:latest",
"Volumes": {
"/prometheus": {}
},
"WorkingDir": "/prometheus",
"Entrypoint": [
"/bin/prometheus"
],
"OnBuild": null,
"Labels": {
"com.docker.compose.config-hash": "f387601dc597da679d6a2290f4192863d07c16791b9d33e29a4e84eb43b2e5df",
"com.docker.compose.container-number": "1",
"com.docker.compose.depends_on": "",
"com.docker.compose.image": "sha256:2a72b385beafd2b3685ad337d14d23f6851f0ff335c5cd9a04c692b3eb846196",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "prometheus",
"com.docker.compose.project.config_files": "/home/lukas/BlackholePort/docker/prometheus/docker-compose.yml",
"com.docker.compose.project.working_dir": "/home/lukas/BlackholePort/docker/prometheus",
"com.docker.compose.service": "prometheus",
"com.docker.compose.version": "2.17.2",
"maintainer": "The Prometheus Authors <[email protected]>"
}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "b882dba1196b5ba11a81cedda028552f453ed82e42cec9fc8fc767d4192571f2",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"9090/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "9090"
},
{
"HostIp": "::",
"HostPort": "9090"
}
]
},
"SandboxKey": "/run/snap.docker/netns/b882dba1196b",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"prometheus_default": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"prometheus",
"prometheus",
"ded87ebe4daf"
],
"NetworkID": "971e8873d46285b63d70fb6feeada747176a5427aa7a19127cb7e8dc8a7d8a37",
"EndpointID": "b67826c8e2736e5ac55ab67a8f9d67a6822dd3260a214622f65de968b0fa972e",
"Gateway": "192.168.32.1",
"IPAddress": "192.168.32.2",
"IPPrefixLen": 20,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:20:02",
"DriverOpts": null
}
}
}
}
]
docker 网络 ls
docker network ls
NETWORK ID NAME DRIVER SCOPE
f124d5db3e0b bridge bridge local
862535ef9371 deluge_default bridge local
86bf6736570f foundryvtt_default bridge local
909b8162cdbe grafana_default bridge local
299ce7db89b5 heimdall_default bridge local
92a81645a176 host host local
a5e580aa1810 microbin_default bridge local
da8d9cb1d3e7 nginx-pm_default bridge local
055308848eba none null local
93afb48bbba2 o-portainer bridge local
460a87dcb670 o-security bridge local
ea16af5d1414 pihole_default bridge local
250440a093d7 portainer_default bridge local
971e8873d462 prometheus_default bridge local
dcbfb120b9b2 prometheus_monitoring bridge local
a482e951879d prowlarr_default bridge local
9ab6f6734815 radarr_default bridge local
f6abd6de38d3 sonarr_default bridge local
0e062c280f80 spigotmc_default bridge local
640505be1145 teamspeak_default bridge local
将 portainer 添加到主机网络后同样的问题
docker-compose.yml
version: '3.8'
volumes:
prometheus_data: {}
services:
prometheus:
image: prom/prometheus:latest
container_name: prometheus
restart: unless-stopped
volumes:
- /home/lukas/data/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
extra_hosts:
- "host.docker.internal:host-gateway"
network_mode: "host"
ports:
- "9090:9090/tcp
docker inspect prometheus（网络部分）
"NetworkSettings": {
"Bridge": "",
"SandboxID": "f114aceaadadca19fe9da85b04dbce4fc9aedae6d7f91a2bed8b3290d3186e82",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {},
"SandboxKey": "/run/snap.docker/netns/default",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"host": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "92a81645a17671cf440926ed83cdb6063790dcf2e18813722d96ccc90614ffb3",
"EndpointID": "4a4f2cef2556314c2feb526928b24784005a4617583e10813bae06f56d5db8e2",
"Gateway": "",
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "",
"DriverOpts": null
}
}
}
答案1
Docker Compose 正在创建一个名为“prometheus_default”的网络，并将 Prometheus 容器连接到该网络。您已将 Docker 守护进程设置为在接口上公布指标，docker0该接口通常位于bridgeDocker 创建的默认网络上。
在运行 Docker 的主机上，您可以通过以下方式验证 Docker 守护程序（-p，显示进程）是否正在侦听（-l）tcp（-t）指标端口号 9323（-n）ss或netstat
$ ss -tlpn | grep :9323
LISTEN 0 4096 127.0.0.1:9323 0.0.0.0:* users:(("dockerd",pid=199176,fd=17))
选项1
尝试将此配置添加到 docker-compose.yaml 以将 Prometheus 添加到桥接网络
version: '3.8'
volumes:
prometheus_data: {}
services:
prometheus:
image: prom/prometheus:latest
container_name: prometheus
restart: unless-stopped
volumes:
- /home/lukas/data/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
extra_hosts:
- "host.docker.internal:host-gateway"
network_mode: bridge
ports:
- "9090:9090/tcp
同时确保 Docker 已设置为在默认网桥上发布指标
{
"metrics-addr": "172.17.0.1:9323"
}
选项 2
更改 Docker 守护进程以监听prometheus_default网络
docker network inspect prometheus_default
[
{
"Name": "prometheus_default",
"Id": "55a40edb42b2f3c1144236a765339e9b1c8dd0542df929430dc07829c5e156d5",
"Created": "2023-03-31T16:51:48.212544059-04:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.19.0.0/16",
"Gateway": "172.19.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {},
"Options": {},
"Labels": {
"com.docker.compose.network": "default",
"com.docker.compose.project": "prometheus",
"com.docker.compose.version": "1.29.2"
}
}
]
在 daemon.json 中设置
{
"metrics-addr": "172.19.0.1:9323"
}
选项 3
允许任何网络访问 Docker 指标（请记住，更多服务将可以访问 Docker 守护进程指标接口，这可能是也可能不是所希望的）
{
"metrics-addr": "0.0.0.0:9323"
}