☰

Centos7主机、KVM pfSense域和Docker在一个盒子里

tag-icon • tag-icon centos docker kvm pfsense
Centos7主机、KVM pfSense域和Docker在一个盒子里

我正在尝试直接在我的服务器上进行路由。

  1. 从 pfSense 域，我可以 ping 管理和 DMZ 网络中的 IP。
  2. 从虚拟主机，我可以 ping 相同的 IP
  3. pfSense 是 DMZ 和管理的网关，我无法从 Centos ping 通 dmz 或管理 GW，
  4. 从 LAN（外部）我可以 ping 通 DMZ 和管理接口的 pfSense 网关。
  5. 使用 pfSense 捕获的数据包显示流量被路由到正确的接口。但没有回程流量。
  6. LAN 到管理是允许任何任何，反之亦然。
  7. 互联网可以工作，所以这就是问题！

盒子里的网络

[root@workhorse1 docker-configs]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vb2 state UP group default qlen 1000
    link/ether 00:21:9b:9d:75:07 brd ff:ff:ff:ff:ff:ff
3: em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vb1 state UP group default qlen 1000
    link/ether 00:21:9b:9d:75:09 brd ff:ff:ff:ff:ff:ff
4: em3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:21:9b:9d:75:0b brd ff:ff:ff:ff:ff:ff
5: em4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:21:9b:9d:75:0d brd ff:ff:ff:ff:ff:ff
10: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:bf:c6:48 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
11: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:bf:c6:48 brd ff:ff:ff:ff:ff:ff
20: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 62:30:db:db:51:bd brd ff:ff:ff:ff:ff:ff
31: vb2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:21:9b:9d:75:07 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.16/24 brd 192.168.1.255 scope global noprefixroute vb2
       valid_lft forever preferred_lft forever
    inet6 fe80::221:9bff:fe9d:7507/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
32: vb1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:21:9b:9d:75:09 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::221:9bff:fe9d:7509/64 scope link 
       valid_lft forever preferred_lft forever
33: vb3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 12:af:3c:dd:a2:72 brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.2/24 scope global vb3
       valid_lft forever preferred_lft forever
    inet6 fe80::7082:e9ff:feed:8e8/64 scope link 
       valid_lft forever preferred_lft forever
34: vb4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:a9:29:c1:18:6c brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.2/24 brd 10.0.0.255 scope global noprefixroute vb4
       valid_lft forever preferred_lft forever
38: vnet0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast master vb1 state DOWN group default qlen 1000
    link/ether 82:f0:9d:32:79:e4 brd ff:ff:ff:ff:ff:ff
40: vnet2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast master vb3 state DOWN group default qlen 1000
    link/ether 12:af:3c:dd:a2:72 brd ff:ff:ff:ff:ff:ff
44: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vb1 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:f8:58:d1 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fef8:58d1/64 scope link 
       valid_lft forever preferred_lft forever
45: vnet5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vb2 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:16:6f:bc brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe16:6fbc/64 scope link 
       valid_lft forever preferred_lft forever
46: vnet6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vb3 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:de:07:95 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fede:795/64 scope link 
       valid_lft forever preferred_lft forever
47: vnet7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vb4 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:7e:d9:ac brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe7e:d9ac/64 scope link 
       valid_lft forever preferred_lft forever
55: docker_gwbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:a7:1a:7f:f4 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker_gwbridge
       valid_lft forever preferred_lft forever
    inet6 fe80::42:a7ff:fe1a:7ff4/64 scope link 
       valid_lft forever preferred_lft forever
63: vethce24fe8@if62: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default 
    link/ether ca:f8:47:76:e5:95 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::c8f8:47ff:fe76:e595/64 scope link 
       valid_lft forever preferred_lft forever
65: br-a5bb2d7cbed8: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:45:9d:7a:7f brd ff:ff:ff:ff:ff:ff
    inet 172.19.0.1/16 brd 172.19.255.255 scope global br-a5bb2d7cbed8
       valid_lft forever preferred_lft forever
351: vethd87b4f6@if350: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default 
    link/ether 9e:0d:17:67:49:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 4
    inet6 fe80::9c0d:17ff:fe67:49c2/64 scope link 
       valid_lft forever preferred_lft forever
151: vnet3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast master vb4 state DOWN group default qlen 1000
    link/ether e2:a9:29:c1:18:6c brd ff:ff:ff:ff:ff:ff
218: vnet1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast master vb2 state DOWN group default qlen 1000
    link/ether b2:68:ec:7a:1a:8a brd ff:ff:ff:ff:ff:ff

[root@workhorse1 docker-configs]# brctl show
bridge name bridge id       STP enabled interfaces
br-a5bb2d7cbed8     8000.0242459d7a7f   no      
docker_gwbridge     8000.0242a71a7ff4   no      veth8f914aa
                            vethce24fe8
vb1     8000.00219b9d7509   yes     em2
                            vnet0
                            vnet4
vb2     8000.00219b9d7507   yes     em1
                            vnet1
                            vnet5
vb3     8000.12af3cdda272   yes     vnet2
                            vnet6
vb4     8000.e2a929c1186c   yes     vnet3
                            vnet7
virbr0      8000.525400bfc648   yes     virbr0-nic

    <interface type='bridge'>
      <mac address='52:54:00:f8:58:d1'/>
      <source bridge='vb1'/>
      <target dev='vnet4'/>
      <model type='e1000'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:16:6f:bc'/>
      <source bridge='vb2'/>
      <target dev='vnet5'/>
      <model type='e1000'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:de:07:95'/>
      <source bridge='vb3'/>
      <target dev='vnet6'/>
      <model type='e1000'/>
      <alias name='net2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:7e:d9:ac'/>
      <source bridge='vb4'/>
      <target dev='vnet7'/>
      <model type='e1000'/>
      <alias name='net3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </interface>

[root@workhorse1 docker-configs]# ip route show
default via 192.168.1.100 dev vb2 proto static metric 431 
10.0.0.0/24 dev vb4 scope link 
172.16.0.0/24 dev vb3 scope link 
172.17.0.0/16 dev docker_gwbridge proto kernel scope link src 172.17.0.1 
172.19.0.0/16 dev br-a5bb2d7cbed8 proto kernel scope link src 172.19.0.1 
192.168.1.0/24 dev vb2 proto kernel scope link src 192.168.1.16 metric 431 
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1

如何通过 pfSense 获得从 LAN 到管理子网的网络连接，然后从 WAN 到 DMZ 的网络连接？

答案1

具有多个接口的计算机，每个接口都有唯一的子网，要求每个接口都有自己的路由表。这是通过基于策略的路由来完成的。

  1. 安装并启用“NetworkManager-config-routing-rules”
  2. 需要设置 defroute=yes 的接口：
root@workhorse1]# more /etc/sysconfig/network-scripts/ifcfg-vb3
DEVICE=vb3
BOOTPROTO=static
ONBOOT=yes
TYPE=Bridge
STP=no
IPADDR=172.16.0.2
NETMASK=255.255.255.0
PREFIX=24
GATEWAY=172.16.0.1
PROXY_METHOD=none
BROWSER_ONLY=no
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
NAME="Bridge vb3"
  1. 向 /etc/iproute2/rt_tables 添加了新的路由表
[root@workhorse1]# more /etc/iproute2/rt_tables
#
# reserved values
#
255 local
254 main
253 default
0   unspec
#
# local
#
#1  inr.ruhep
1   lan
2   mgmt
3   dmz
  1. 添加默认路由：
[root@workhorse1]# more /etc/sysconfig/network-scripts/route-vb2
192.168.1.0/24 dev vb2 table lan
default via 192.168.1.100 dev vb2 table lan
[root@workhorse1]# more /etc/sysconfig/network-scripts/route-vb3
172.16.0.0/24 dev vb3 table mgmt
default via 172.16.0.1 dev vb3 table mgmt
[root@workhorse1]# more /etc/sysconfig/network-scripts/route-vb4
10.0.0.0/24 dev vb4 table dmz
default via 10.0.0.1 dev vb4 table dmz
  1. 新增路由策略：
[root@workhorse1]# more /etc/sysconfig/network-scripts/rule-vb2
from 192.168.1.0/24 lookup lan
[root@workhorse1]# more /etc/sysconfig/network-scripts/rule-vb3
from 172.16.0.0/24 lookup mgmt
[root@workhorse1]# more /etc/sysconfig/network-scripts/rule-vb4
from 10.0.0.2/24 lookup dmz
  1. 技术上不需要，但我重新启动了整个服务器。
  2. 验证路线：
[root@workhorse1 vm]# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.1.100   0.0.0.0         UG        0 0          0 vb2
0.0.0.0         10.0.0.1        0.0.0.0         UG        0 0          0 vb4
0.0.0.0         172.16.0.1      0.0.0.0         UG        0 0          0 vb3
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0 vb4
172.16.0.0      0.0.0.0         255.255.255.0   U         0 0          0 vb3
172.17.0.0      0.0.0.0         255.255.0.0     U         0 0          0 docker_gwbridge
172.19.0.0      0.0.0.0         255.255.0.0     U         0 0          0 br-a5bb2d7cbed8
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 vb2
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 virbr0

**故障排除：使用tcpdump和pfsense抓包查看流量。

**旁注，pfSense 阻止了我从主机到其各个接口的 ping。所以，这是我的一个错误。

相关内容