我正在尝试创建新的 docker 网络桥接器，通过我的第二个网络接口路由传出流量，我设法使容器流量通过所需的接口，但是我失去了从本地主机与容器通信的能力，同样也无法从容器内部通信。这是迄今为止的配置。
我的网络接口：
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.0.3 netmask 255.255.255.0 broadcast 10.0.0.255
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.1.3 netmask 255.255.255.0 broadcast 10.0.1.255
docker_backup: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.18.0.1 netmask 255.255.0.0 broadcast 172.18.255.255
Docker 网桥配置
[
{
"Name": "docker_backup",
"Id": "5252a641d2087e5d4ed6aa142327a3193c2142d8fe5cac6b6a163d0154819d6d",
"Created": "2021-11-19T16:36:37.037180993+03:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "172.18.0.0/16",
"Gateway": "172.18.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {},
"Options": {
"com.docker.network.bridge.enable_ip_masquerade": "false",
"com.docker.network.bridge.name": "docker_backup"
},
"Labels": {}
}
]
猫/等/iproute2/rt_tables
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep
1 docker_backup
iptables
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0
SNAT all -- 172.18.0.0/16 0.0.0.0/0 to:10.0.1.3
MASQUERADE all -- 172.19.0.0/16 0.0.0.0/0
MASQUERADE all -- 172.23.0.0/16 0.0.0.0/0
MASQUERADE tcp -- 172.18.0.3 172.18.0.3 tcp dpt:80
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
DNAT tcp -- 0.0.0.0/0 127.0.0.1 tcp dpt:8081 to:172.18.0.3:80
路由和 IP 表
ip rule add from 172.18.0.0/16 tab docker_backup
ip route add 172.18.0.0/16 dev eth1 tab docker_backup
ip route add default via 10.0.1.1 dev eth1 tab docker_backup
iptables -t nat -A POSTROUTING -s 172.18.0.0/16 ! -o docker_backup -j SNAT --to-source 10.0.1.3
netstat -an | grep ‘听’
# netstat -an | grep 'LISTEN '
tcp 0 0 127.0.0.1:3493 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:139 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:8081 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:5938 0.0.0.0:* LISTEN
tcp 0 0 10.0.1.3:8083 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:5941 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:445 0.0.0.0:* LISTEN
tcp 0 0 10.0.1.3:6881 0.0.0.0:* LISTEN
路线
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default homerouter.cpe 0.0.0.0 UG 100 0 0 eth0
10.0.0.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
10.0.1.0 0.0.0.0 255.255.255.0 U 101 0 0 eth1
link-local 0.0.0.0 255.255.0.0 U 1000 0 0 eth1
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker_backup
172.19.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-3271a2013d45
172.23.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-1855923fbb3f
docker-compose 示例
version: "3.8"
services:
tester:
image: linuxserver/nginx
environment:
- TZ=Asia/Kuwait
volumes:
- ./config:/config
ports:
- 127.0.0.1:8081:80
networks:
docker_backup:
ipv4_address: 172.18.0.3
networks:
docker_backup:
external: true
从堡垒主机 curl 尝试：
(07:27:55) [email protected] ~/tests
# curl -v localhost:8081
* Trying 127.0.0.1:8081...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8081 (#0)
> GET / HTTP/1.1
> Host: localhost:8081
> User-Agent: curl/7.68.0
> Accept: */*
>
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer
(07:28:10) [email protected] ~/tests
# curl -v 172.18.0.3:80
* Trying 172.18.0.3:80...
* TCP_NODELAY set
* connect to 172.18.0.3 port 80 failed: No route to host
* Failed to connect to 172.18.0.3 port 80: No route to host
* Closing connection 0
curl: (7) Failed to connect to 172.18.0.3 port 80: No route to host
从容器内部我可以使用 eth1 IP 地址连接到外部世界
root@50ed3e196cde:/# curl -LSs ifconfig.io
37.xx.xxx.xx
路由 & ifconfig & ping 到容器网关。
root@434d48b82d6a:/# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default server 0.0.0.0 UG 0 0 0 eth0
172.18.0.0 * 255.255.0.0 U 0 0 0 eth0
root@434d48b82d6a:/# ping -W 5 172.18.0.1
PING 172.18.0.1 (172.18.0.1): 56 data bytes
--- 172.18.0.1 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss
root@434d48b82d6a:/# ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:AC:12:00:03
inet addr:172.18.0.3 Bcast:172.18.255.255 Mask:255.255.0.0
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
但是，任何 ping 其他 docker 或主主机 ip（即本地地址）的尝试都会失败。
root@50ed3e196cde:/# curl -v --connect-timeout 5 172.18.0.1
* Trying 172.18.0.1:80...
* After 5000ms connect time, move on!
* connect to 172.18.0.1 port 80 failed: Operation timed out
* Connection timeout after 5000 ms
* Closing connection 0
curl: (28) Connection timeout after 5000 ms
我可以从主主机 ping 容器 ip，但无法访问任何公开的服务。
# ping 172.18.0.3
PING 172.18.0.3 (172.18.0.3) 56(84) bytes of data.
64 bytes from 172.18.0.3: icmp_seq=1 ttl=64 time=0.095 ms
64 bytes from 172.18.0.3: icmp_seq=2 ttl=64 time=0.074 ms
64 bytes from 172.18.0.3: icmp_seq=3 ttl=64 time=0.092 ms
64 bytes from 172.18.0.3: icmp_seq=4 ttl=64 time=0.086 ms
64 bytes from 172.18.0.3: icmp_seq=5 ttl=64 time=0.087 ms
64 bytes from 172.18.0.3: icmp_seq=6 ttl=64 time=0.089 ms
^C
--- 172.18.0.3 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5098ms
rtt min/avg/max/mdev = 0.074/0.087/0.095/0.006 ms
(07:10:15) [email protected] ~/tests
# ping 172.18.0.3^C
(07:10:19) [email protected] ~/tests
# curl -v http://172.18.0.3:80
* Trying 172.18.0.3:80...
* TCP_NODELAY set
* connect to 172.18.0.3 port 80 failed: No route to host
* Failed to connect to 172.18.0.3 port 80: No route to host
* Closing connection 0
curl: (7) Failed to connect to 172.18.0.3 port 80: No route to host
所以，问题是我该如何修改路由或 iptables 规则以使流量从本地地址流向容器，并从容器流向本地地址，同时保持传出的 ip？
答案1
我通过在主主机上添加新的 IP 规则来修复此问题，该规则将到容器子网的流量路由到主表中。
ip rule add to 172.18.0.0/16 lookup main priority 10