我们正在尝试在两台不同的机器上设置双节点 Hadoop 集群。两台机器上都已设置了无需密码的 SSH 到本地主机。Hadoop 可以在每台单独的机器上作为单节点运行良好。
我们正处于使用 ssh 连接到对方服务器并尝试将公钥复制到对方的 .ssh 文件夹的阶段。但问题是我们似乎甚至无法通过 ip 找到服务器:
这是我尝试 ssh 到另一台机器时得到的结果:
ssh -v jingfang@hadoop-master
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to hadoop-master [192.168.1.52] port 22.
debug1: connect to address 192.168.1.52 port 22: Resource temporarily unavailable
ssh: connect to host hadoop-master port 22: Resource temporarily unavailable
另一台机器尝试连接我时也得到了类似的响应。我们尝试 ssh 到其他公共服务器,比如学校的服务器,我们可以登录。使用 putty 时,我可以使用我提供给对方的 IP 地址连接到我自己的机器,但无法 ssh 到对方。而对方可以用同样的方式 putty 到自己的机器,但无法 putty 到我的机器。
我们认为要么是因为 ssh 服务器配置不正确,要么是网络配置错误。这是我的 ssh_config
# This is the ssh client system-wide configuration file. See
# ssh_config(5) for more information. This file provides defaults for
# users, and the values can be changed in per-user configuration files
# or on the command line.
# Configuration data is parsed as follows:
# 1. command line options
# 2. user-specific file
# 3. system-wide file
# Any configuration value is only changed the first time it is set.
# Thus, host-specific definitions should be at the beginning of the
# configuration file, and defaults at the end.
# Site-wide defaults for some commonly used options. For a comprehensive
# list of available options, their meanings and defaults, please see the
# ssh_config(5) man page.
Host *
# ForwardAgent no
# ForwardX11 no
# ForwardX11Trusted yes
# PasswordAuthentication yes
# HostbasedAuthentication no
# GSSAPIAuthentication no
# GSSAPIDelegateCredentials no
# GSSAPIKeyExchange no
# GSSAPITrustDNS no
# BatchMode no
CheckHostIP yes
AddressFamily any
# ConnectTimeout 0
StrictHostKeyChecking ask
IdentityFile /home/hadoop/.ssh/id_rsa
# IdentityFile /home/hadoop/.ssh/id_dsa
# IdentityFile /home/hadoop/.ssh/id_ecdsa
# IdentityFile /home/hadoop/.ssh/id_ed25519
Port 22
Protocol 2
# Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc
# MACs hmac-md5,hmac-sha1,[email protected]
# EscapeChar ~
# Tunnel no
# TunnelDevice any:any
# PermitLocalCommand no
# VisualHostKey no
# ProxyCommand ssh -q -W %h:%p gateway.example.com
# RekeyLimit 1G 1h
SendEnv LANG LC_*
HashKnownHosts yes
GSSAPIAuthentication yes
这是我的 sshd_config:
# $OpenBSD: sshd_config,v 1.101 2017/03/14 07:19:07 djm Exp $
# This is the sshd server system-wide configuration file. See
# sshd_config(5) for more information.
# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin
# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented. Uncommented options override the
# default value.
Port 22
AddressFamily any
ListenAddress 0.0.0.0
#ListenAddress ::
HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key
# Ciphers and keying
#RekeyLimit default none
# Logging
#SyslogFacility AUTH
#LogLevel INFO
# Authentication:
#LoginGraceTime 2m
PermitRootLogin without-password
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10
PubkeyAuthentication yes
# Expect .ssh/authorized_keys2 to be disregarded by default in future.
AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2
#AuthorizedPrincipalsFile none
#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody
# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes
# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no
# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no
# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no
# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes
#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none
# no default banner path
#Banner none
# Allow client to pass locale environment variables
AcceptEnv LANG LC_*
# override default of no subsystems
Subsystem sftp /usr/lib/openssh/sftp-server
# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# PermitTTY no
# ForceCommand cvs server
请建议我们是否应该更改任何 ssh/sshd 配置或网络配置。非常感谢!
答案1
您需要知道通过其连接到互联网的每个位置的公共 IP,然后需要在每个位置的路由器上设置端口转发,以将端口 22 发送到每个位置所需的计算机的 IP。完成后,通过 ssh 连接到另一台计算机的公共 IP,确保两台计算机上都设置了密码验证,并且每台计算机都有一个生成的 RSA 密钥。然后,您可以ssh-copy-id username@somehost
将密钥从每台计算机复制到另一台计算机。之后,禁用密码验证并启用公钥验证。然后重新启动 ssh。如果您知道需要在端口转发中使用该端口而不是 22,我也会考虑使用非标准端口。
答案2
我认为对网络如何工作(或不工作)的一些了解可能是一个开始。
本质上 - 从您的描述来看,您在 NAT 类型设置中有两台机器。现在假设其中一台或多台不在 CGN 后面(这会使事情迅速复杂化)。
现在你的网络看起来有点像这样
School Server
^
|
|
|
v
+--------> Internet <----+
| |
| |
| |
| |
v v
+---+---+ +--+----<---+
+----> |Router?| |Router? |
| | NAT A| |NAT B | |
+ +-------+ +------+ +
+-------------+ +---------------+
| Endpoint A | | Endpoint B |
| | | |
+-------------+ +---------------+
有几点需要注意。如果您的端点位于 NAT 后面,则可以启动与更广泛互联网上的系统的连接。但是端点 B 不知道如何路由到端点 A,因为它们位于 NAT 后面 - 并且连接出去被伪装,反之亦然。但是他们会知道如何路由到学校服务器,因为那一端的路由器能够路由到互联网。
这就是为什么端点 A 和 B 可以看到学校服务器的原因(因为它们的流量可以被路由出去),但他们彼此无法看到对方(因为他们的 IP 对彼此来说毫无意义)。
选择如下
- 如果可能的话,在路由器上进行端口转发,并设置 ssh 以使用这些端口(如果不是标准端口)。
这是最簡單但假设两个端点都是可路由的,并且不在 NAT 或运营商级 NAT 之后,并且您有能力打开端口
- 设置某种 VPN。这至少需要至少您可以对一个系统进行一定的控制,并且两个系统都可以访问它。星云较新,但似乎很合适。然后您可以将这些系统视为位于同一局域网上
也可以看看零层- 相同的想法,将涉及的机器放在同一个局域网上。这应该更简单,但你依赖的是外部服务。
- 设置 IPv6 隧道是另一种选择。这更复杂,需要相当多的技能但一旦完成,就会很好地扩展,并且由于您可以通过 ipv6 发送流量,因此您可以简单地从一个客户端连接到另一个客户端。
飓风电气是首选供应商,尽管这可能是三者中技术难度最大的选择。