RHEL ec2 实例上额外 eni 的 ip 路由

RHEL ec2 实例上额外 eni 的 ip 路由

我注意到,当在 ec2 上添加额外的 ENI 时,使路由正常工作非常困难,并且没有任何文档可供参考。

我确实知道,如果有人使用 Amazon Linux ami,一切无需额外工作即可自动运行。但使用任何其他 ami,必须配置路由才能运行。

非 Amazon Linux

[root@ip-10-11-2-20 ~]# cat /etc/*release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.3 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.3"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.3 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.3:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.3
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.3"
Red Hat Enterprise Linux Server release 7.3 (Maipo)
Red Hat Enterprise Linux Server release 7.3 (Maipo)

[root@ip-10-11-2-20 ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.11.2.1       0.0.0.0         UG    100    0        0 eth0
0.0.0.0         10.11.2.1       0.0.0.0         UG    101    0        0 eth1
10.11.2.0       0.0.0.0         255.255.255.0   U     100    0        0 eth0
10.11.2.0       0.0.0.0         255.255.255.0   U     101    0        0 eth1
[root@ip-10-11-2-20 ~]# ip route
default via 10.11.2.1 dev eth0  proto static  metric 100
default via 10.11.2.1 dev eth1  proto static  metric 101
10.11.2.0/24 dev eth0  proto kernel  scope link  src 10.11.2.20  metric 100
10.11.2.0/24 dev eth1  proto kernel  scope link  src 10.11.2.247  metric 101

[root@ip-10-11-2-20 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 10.11.2.20  netmask 255.255.255.0  broadcast 10.11.2.255
        inet6 fe80::1012:30ff:feca:4bd0  prefixlen 64  scopeid 0x20<link>
        ether 12:12:30:ca:4b:d0  txqueuelen 1000  (Ethernet)
        RX packets 3065  bytes 258041 (251.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2375  bytes 351169 (342.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 10.11.2.247  netmask 255.255.255.0  broadcast 10.11.2.255
        inet6 fe80::1034:1bff:fe8d:c742  prefixlen 64  scopeid 0x20<link>
        ether 12:34:1b:8d:c7:42  txqueuelen 1000  (Ethernet)
        RX packets 473  bytes 22008 (21.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 58  bytes 5544 (5.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 140  bytes 11924 (11.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 140  bytes 11924 (11.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@ip-10-11-2-20 ~]# ping -c 3 10.11.2.20
PING 10.11.2.20 (10.11.2.20) 56(84) bytes of data.
64 bytes from 10.11.2.20: icmp_seq=1 ttl=64 time=0.013 ms
64 bytes from 10.11.2.20: icmp_seq=2 ttl=64 time=0.027 ms
64 bytes from 10.11.2.20: icmp_seq=3 ttl=64 time=0.025 ms

--- 10.11.2.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.013/0.021/0.027/0.008 ms

[root@ip-10-11-2-20 ~]# ping -c 3 10.11.2.247
PING 10.11.2.247 (10.11.2.247) 56(84) bytes of data.
64 bytes from 10.11.2.247: icmp_seq=1 ttl=64 time=0.013 ms
64 bytes from 10.11.2.247: icmp_seq=2 ttl=64 time=0.024 ms
64 bytes from 10.11.2.247: icmp_seq=3 ttl=64 time=0.024 ms

--- 10.11.2.247 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.013/0.020/0.024/0.006 ms

亚马逊 Linux

[root@ip-10-11-2-149 ~]# cat  /etc/*release
NAME="Amazon Linux AMI"
VERSION="2017.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2017.03"
PRETTY_NAME="Amazon Linux AMI 2017.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2017.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
Amazon Linux AMI release 2017.03

[root@ip-10-11-2-149 ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.11.2.1       0.0.0.0         UG    0      0        0 eth0
0.0.0.0         10.11.2.1       0.0.0.0         UG    10001  0        0 eth1
10.11.2.0       0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.11.2.0       0.0.0.0         255.255.255.0   U     0      0        0 eth1
169.254.169.254 0.0.0.0         255.255.255.255 UH    0      0        0 eth0

[root@ip-10-11-2-149 ~]# ip route
default via 10.11.2.1 dev eth0
default via 10.11.2.1 dev eth1  metric 10001
10.11.2.0/24 dev eth0  proto kernel  scope link  src 10.11.2.149
10.11.2.0/24 dev eth1  proto kernel  scope link  src 10.11.2.61
169.254.169.254 dev eth0

[root@ip-10-11-2-149 ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 12:B2:B8:77:D0:F8
          inet addr:10.11.2.149  Bcast:10.11.2.255  Mask:255.255.255.0
          inet6 addr: fe80::10b2:b8ff:fe77:d0f8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9001  Metric:1
          RX packets:1490 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1441 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:129285 (126.2 KiB)  TX bytes:143368 (140.0 KiB)

eth1      Link encap:Ethernet  HWaddr 12:9B:45:32:EB:BA
          inet addr:10.11.2.61  Bcast:10.11.2.255  Mask:255.255.255.0
          inet6 addr: fe80::109b:45ff:fe32:ebba/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9001  Metric:1
          RX packets:263 errors:0 dropped:0 overruns:0 frame:0
          TX packets:347 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:16825 (16.4 KiB)  TX bytes:25705 (25.1 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:2 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:140 (140.0 b)  TX bytes:140 (140.0 b)

[root@ip-10-11-2-149 ~]# ping -c 3 10.11.2.149
PING 10.11.2.149 (10.11.2.149) 56(84) bytes of data.
64 bytes from 10.11.2.149: icmp_seq=1 ttl=255 time=0.018 ms
64 bytes from 10.11.2.149: icmp_seq=2 ttl=255 time=0.027 ms
64 bytes from 10.11.2.149: icmp_seq=3 ttl=255 time=0.026 ms

--- 10.11.2.149 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2035ms
rtt min/avg/max/mdev = 0.018/0.023/0.027/0.006 ms

[root@ip-10-11-2-149 ~]# ping -c 3 10.11.2.61
PING 10.11.2.61 (10.11.2.61) 56(84) bytes of data.
64 bytes from 10.11.2.61: icmp_seq=1 ttl=255 time=0.018 ms
64 bytes from 10.11.2.61: icmp_seq=2 ttl=255 time=0.027 ms
64 bytes from 10.11.2.61: icmp_seq=3 ttl=255 time=0.029 ms

--- 10.11.2.61 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2034ms
rtt min/avg/max/mdev = 0.018/0.024/0.029/0.007 ms

现在问题就出在下面...看看非亚马逊 Linux 如何能够 ping 通亚马逊 Linux 上的 ENI,但亚马逊 Linux 却无法 ping 通eth0非亚马逊 Linuxeth1eth1

INTER NETWORKING

非 Amazon Linux

[root@ip-10-11-2-20 ~]# ping -c 3 10.11.2.149
PING 10.11.2.149 (10.11.2.149) 56(84) bytes of data.
64 bytes from 10.11.2.149: icmp_seq=1 ttl=255 time=0.629 ms
64 bytes from 10.11.2.149: icmp_seq=2 ttl=255 time=0.486 ms
64 bytes from 10.11.2.149: icmp_seq=3 ttl=255 time=0.472 ms

--- 10.11.2.149 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.472/0.529/0.629/0.070 ms

[root@ip-10-11-2-20 ~]# ping -c 3 10.11.2.61
PING 10.11.2.61 (10.11.2.61) 56(84) bytes of data.
64 bytes from 10.11.2.61: icmp_seq=1 ttl=255 time=0.595 ms
64 bytes from 10.11.2.61: icmp_seq=2 ttl=255 time=0.560 ms
64 bytes from 10.11.2.61: icmp_seq=3 ttl=255 time=0.522 ms

--- 10.11.2.61 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.522/0.559/0.595/0.029 ms

亚马逊 Linux

[root@ip-10-11-2-149 ~]# ping -c 3 10.11.2.20
PING 10.11.2.20 (10.11.2.20) 56(84) bytes of data.
64 bytes from 10.11.2.20: icmp_seq=1 ttl=64 time=0.446 ms
64 bytes from 10.11.2.20: icmp_seq=2 ttl=64 time=0.465 ms
64 bytes from 10.11.2.20: icmp_seq=3 ttl=64 time=0.481 ms

--- 10.11.2.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2030ms
rtt min/avg/max/mdev = 0.446/0.464/0.481/0.014 ms

[root@ip-10-11-2-149 ~]# ping -c 3 10.11.2.247
PING 10.11.2.247 (10.11.2.247) 56(84) bytes of data.
^C
--- 10.11.2.247 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2028ms

我如何允许路由到eth1添加到 RHEL 7 ec2 实例的额外 ENI?

答案1

我以前遇到过这个问题,后来发现了这个博客文章有帮助

关于路由设置不正确的问题。我已将其添加到 Userdata 脚本中,因此在启动时路由会正确添加。假设我们在 eth1 上附加了一个新的 ENI。

# Use the AWS CLI to get the id of the ENI to be attached
  NETWORK_INTERFACE_ID=`aws ec2 describe-network-interfaces --filters "Name=status,Values=available" "Name=tag:Name,Values=$ENI_NAME" "Name=subnet-id,Values=$SUBNET_ID" --output json --query "NetworkInterfaces[0].NetworkInterfaceId" | grep -o 'eni-[a-z0-9]*'`
  # Attach the ENI (and display the attachment id)
  echo "Attaching ENI:$ENI_NAME  NETID:$NETWORK_INTERFACE_ID Index:$ENI_INDEX"

  aws ec2 attach-network-interface --network-interface-id $NETWORK_INTERFACE_ID --instance-id $SELF_INSTANCE_ID --device-index $ENI_INDEX


  printf '\nauto eth1\niface eth1 inet dhcp' >> /etc/network/interfaces.d/eth1.cfg

  GATEWAY=`ip route | awk '/default/ { print $3 }'`
  #get the secondary ip
  SECOND_IP=`aws ec2 describe-network-interfaces --filters  "Name=tag:Name,Values=$ENI_NAME" "Name=subnet-id,Values=$SUBNET_ID" --output text --query "NetworkInterfaces[0].PrivateIpAddress"`
  #add the routes



  echo "Adding the IP routes and rules for secondary $SECOND_IP"
  printf "\nup ip route add default via $GATEWAY dev eth1 tab 2\n" >> /etc/network/interfaces.d/eth1.cfg
  printf "up ip rule add from $SECOND_IP/32 tab 2 prio 1000\n" >> /etc/network/interfaces.d/eth1.cfg

  sleep 20 # the attach takes some time to happen

  systemctl restart networking.service

  ifup eth1

  ip rule list

  ifconfig

答案2

只需补充一下我所做的事情,事情就开始起作用了

刚刚从@strongjz提到的URL中添加了以下4行

非 Amazon Linux

ip route add default via 10.11.2.1 dev eth0 tab 1
ip route add default via 10.11.2.1 dev eth1 tab 2

ip rule add from 10.11.2.20/32 tab 1 priority 500
ip rule add from 10.11.2.247/32 tab 2 priority 600

剩下的就是历史了...所有 ping 操作都可以对所有接口起作用,包括 eth0 和 eth1!!!

这一切都归功于这个 URLhttps://forums.aws.amazon.com/message.jspa?messageID=404398

相关内容