我正在执行 SFTP 以将文件放入新的外部客户端。我们还有大约 10 个其他客户端,一些是内部的,一些是外部的,我们可以毫无问题地上传到这些客户端。这个新客户端与我们一起使用 VPN(我们的一些其他客户端也与我们一起使用 VPN,但不是全部),在传输 50M 文件中的大约 40M 后,它会以 EOF 响应 put 命令。在过去的 14 次执行中,有 10 次发生了这种情况。
除了两次之外,在当天的第一次运行中,大约凌晨 5:00 时,只收到一次 EOF。几分钟后我再次运行 SFTP 时,put 命令起作用了,并且传输了完整的文件。在当天的剩余时间里,它也起作用了。在同样的执行中,我们还发送了另外两个文件,一个是 1M,另一个是 10M。我的脚本按文件大小对文件列表进行排序,因此较小的文件会先发送。所有这些都在同一个连接中。我的脚本首先连接,发送密码并确保它被接受,然后执行更改目录、ls 命令和 df 命令。然后脚本中有一个循环,用于放置列出的所有文件。前两个文件放置成功,然后放置第三个文件,但因 EOF 而失败。
有两次例外,第二次尝试时我收到了 EOF,第三次尝试时 put 成功。总共,这是最后 14 次运行中的 10 次。其他四次运行在第一次(当天)成功。这项工作大约在同一时间运行(4:50 到 5:15)。我已经与客户进行了交谈,并稍微更改了时间以避开他们运行文件夹监视过程的间隔。但这并没有什么区别。
我们在 AIX 7.1 上运行 OpenSSH_6.0p1,他们在 CentOS Linux 7.4 上运行 OpenSSH_7.4p1。我曾两次尝试使用详细选项 (-vvv) 从命令行运行 SFTP,这两次都成功了。我将更新我的脚本,以某种方式将详细输出捕获到日志文件中。但我在脚本和命令行中都得到了这个 EOF,所以我不认为这是脚本的问题。我可以先发送另外两个文件,所以这不仅仅是发送的第一个文件的问题。这不仅仅是文件大小的问题,即使它只在 50M 文件上失败,因为几分钟后它就可以正常工作了。
所以我的一些问题是 - 什么会导致 put 操作出现 EOF?我找到了 SFTP 的版本 2 RFC 规范,我没有看到关于 put 操作出现 EOF 的提及,只有 get 操作出现 EOF。CentOS Linux 上是否有一些后台进程会导致大文件传输出现 EOF?传输大约需要 25 秒。这可能是网络错误吗?网络质量一开始可能很差,但几分钟后会自动升级到更高质量的水平吗?客户端距离 2700 英里。感谢您的任何评论。Peter
更新:供应商向我发送了他们的 SFTP 调试日志,但我没有发现任何具体原因:
28745 debug1: request 786: write "/upload/X_Y_Z_file_20180315040147.txt" (handle 0) off 25329664 len 32768
28745 debug3: request 786: sent status 0
28745 sent status Success
28745 debug1: request 787: write "/upload/X_Y_Z_file_20180315040147.txt" (handle 0) off 25362432 len 32768
28745 debug3: request 787: sent status 0
28745 sent status Success
28745 debug1: request 788: write "/upload/X_Y_Z_file_20180315040147.txt" (handle 0) off 25395200 len 32768
28745 debug3: request 788: sent status 0
28745 sent status Success
28745 debug1: read eof
28745 forced close "/upload/X_Y_Z_file_20180315040147.txt" bytes read 0 written 25427968
28745 session closed for local user abc from [1.2.3.4]
编辑:我不认为我的脚本是问题所在(但我知道这听起来怎么样),因为无论我使用脚本还是从命令行运行 SFTP 命令,我都会遇到同样的问题。该脚本使用 EXPECT 来捕获 SFTP 响应,该响应是“100%”或 EOF 或 TIMEOUT 之一。一旦我弄清楚如何做到这一点,我就会附上脚本。
编辑:这是 sftp put 的脚本:
#! /usr/bin/ksh
# The following line is seen as a continuecomment by Tcl\
exec $QUOVADX_INSTALL_DIR/integrator/bin/hcitcl "$0" ${1+"$@"}
# subroutines first
proc gts {} {
set tt [clock format [clock seconds] -format {%y/%m/%d %H:%M:%S} ]
set ms [format %03d [expr {[clock clicks -milliseconds] % 1000}]]
return $tt.$ms
}
# proc to write to debug file
proc debugw {msg} {
global debugfile
set fh [open $debugfile a] ; puts $fh "$msg" ; close $fh
}
set frdir [lindex $argv 0]
set lfile [lindex $argv 1]
set todir [lindex $argv 2]
set site [lindex $argv 3]
set debug $::env(cldebug)
set parmfile $::env(clparmfile)
set module "sftp_put_list"
# get debugfilename
if {$debug} {set debugfile [exec getdebugfile.tcl] ; global debugfile}
if {$debug} {debugw "[gts] $module start"}
if {$debug} {debugw "[gts] $module processing files in list $lfile"}
# get SFTP connection parms from parmfile using $site as key
set hfile [open $parmfile r] ; set data [read $hfile] ; close $hfile
set lList [split $data "\n"] ; set pos [lsearch -regexp $lList "^key_sftp_$site"]
lassign [split [lindex $lList $pos] " "] key port url pass
if {$debug > 1} {debugw "[gts] $module key: $key port $port url $url pass $pass"}
# url with vertical bar means the first part is the userid and second is the url
set userList [split "$url" "|"] ; lassign $userList user url
if {$debug} {debugw "[gts] $module userList: $userList"}
set ullen [llength $userList]
if {$debug} {debugw "[gts] $module userList length: $ullen"}
log_user 0
# start/spawn the sftp process - either simple user url or separate user
if {[llength $userList] eq 1} {
set hname "$user"
spawn /usr/bin/sftp -o Port=$port -o ConnectTimeout=60 $user
if {$debug} {debugw "[gts] $module connection uses simple user url $user"}
} else {
set hname "$url"
spawn /usr/bin/sftp -o Port=$port -o ConnectTimeout=60 -o User=$user $url
if {$debug} {debugw "[gts] $module connection uses separate user $user and url $url"}
}
set pass_word_prompt "*?assword:*"
set sftp_prompt "*?ftp>"
set timeout 300
set sftp_put_timeout 1000
set prompt_timeout 300
#--------------------------------------------------------------
# first, wait for the password prompt - if a password is required!
#--------------------------------------------------------------
if {$pass ne "#"} {
expect {
eof {
echo "sftp EOF waiting for password prompt|NA"
if {$debug} {debugw "[gts] $module EOF received waiting for password prompt from $hname - exit 8"}
exit 8
}
"$pass_word_prompt" {
if {$debug} {debugw "[gts] $module password prompt received and answered"}
}
timeout {
echo "sftp timeout waiting for password prompt|NA"
if {$debug} {debugw "[gts] $module timeout waiting for password prompt from $hname - exit 8"}
exit 8
}
}
set timeout $prompt_timeout
send "$pass\r"
expect {
eof {
echo "sftp EOF waiting for command prompt after entering password|NA"
if {$debug} {debugw "[gts] $module EOF received waiting for command prompt from $hname after entering password - exit 8"}
exit 8
}
"$sftp_prompt" {
if {$debug} {debugw "[gts] $module received command prompt; will enter LCD command"}
}
timeout {
echo "sftp timeout after $prompt_timeout seconds waiting for command prompt after entering password|NA"
if {$debug} {debugw "[gts] $module timeout after $prompt_timeout seconds waiting for command prompt from $hname after entering password - exit 8"}
exit 8
}
}
} else {
expect {
eof {
echo "sftp EOF waiting for command prompt after logging on|NA"
if {$debug} {debugw "[gts] $module EOF received waiting for command prompt from $hname after logging on - exit 8"}
exit 8
}
"$sftp_prompt" {
if {$debug} {debugw "[gts] $module received command prompt; will enter LCD command"}
}
timeout {
echo "sftp timeout after $prompt_timeout seconds waiting for command prompt after logging on|NA"
if {$debug} {debugw "[gts] $module timeout after $prompt_timeout seconds waiting for command prompt from $hname after logging on - exit 8"}
exit 8
}
}
}
#--------------------------------------------------------------
# lcd to local directory from where you will send the file(s)
#--------------------------------------------------------------
set timeout $prompt_timeout
send "lcd $frdir\r"
expect {
eof {
echo "sftp EOF waiting for command prompt after entering LCD|NA"
if {$debug} {debugw "[gts] $module EOF received waiting for command prompt from $hname after entering LCD - exit 8"}
exit 8
}
"$sftp_prompt" {
if {$debug} {debugw "[gts] $module received command prompt after LCD ; will enter CD command"}
}
timeout {
echo "sftp timeout after $prompt_timeout seconds waiting for command prompt after entering LCD|NA"
if {$debug} {debugw "[gts] $module timeout after $prompt_timeout seconds waiting for command prompt from $hname after entering LCD - exit 8"}
exit 8
}
}
#--------------------------------------------------------------
# cd to remote directory where you will put the file(s)
#--------------------------------------------------------------
set timeout $prompt_timeout
send "cd $todir\r"
expect {
eof {
echo "sftp EOF waiting for command prompt after entering CD command|NA"
if {$debug} {debugw "[gts] $module EOF received waiting for prompt from $hname after entering CD command - exit 8"}
exit 8
}
"No such file or directory" {
echo "remote directory $todir not found or not accessible|NA"
if {$debug} {debugw "[gts] $module remote directory $todir not found or not accessible at $hname - exit 8"}
exit 8
}
"$sftp_prompt" {
if {$debug} {debugw "[gts] $module received command prompt after CD; will execute PWD command"}
}
timeout {
echo "sftp timeout after $prompt_timeout seconds waiting for prompt after entering CD command|NA"
if {$debug} {debugw "[gts] $module timeout after $prompt_timeou seconds waiting for command prompt from $hname after entering CD command - exit 8"}
exit 8
}
}
#--------------------------------------------------------------
# print working directory
#--------------------------------------------------------------
set timeout $prompt_timeout
send "pwd\r"
expect {
eof {
echo "sftp EOF waiting for command prompt after entering pwd command|NA"
if {$debug} {debugw "[gts] $module EOF received waiting for prompt from $hname after entering pwd command - exit 8"}
exit 8
}
"$sftp_prompt" {
set response $expect_out(0,string)
set rwd [lindex [split "$response" ":"] 1]
set rwd [lindex [split "$rwd" "\r" ] 0]
if {$debug} {debugw "[gts] $module pwd: $rwd"}
if {$debug} {debugw "[gts] $module received command prompt after PWD; will execute DIR command"}
}
timeout {
echo "sftp timeout after $prompt_timeout seconds waiting for prompt after entering pwd command|NA"
if {$debug} {debugw "[gts] $module timeout after $prompt_timeout seconds waiting for command prompt from $hname after entering pwd command - exit 8"}
exit 8
}
}
#--------------------------------------------------------------
# directory list
#--------------------------------------------------------------
set timeout $prompt_timeout
send "ls -l\r"
expect {
eof {
echo "sftp EOF waiting for command prompt after entering dir command|NA"
if {$debug} {debugw "[gts] $module EOF received waiting for prompt from $hname after entering dir command - exit 8"}
exit 8
}
"$sftp_prompt" {
set response $expect_out(0,string)
set dir [lrange [split "$response" "\r"] 1 end-1]
set dir [string map {\{ "" \} ""} "$dir"]
set dir [string trim "$dir"]
if {$debug} {debugw "[gts] $module dir: \r$dir"}
if {$debug} {debugw "[gts] $module received command prompt after DIR; will execute PUT command"}
}
timeout {
echo "sftp timeout after $prompt_timeout seconds waiting for prompt after entering dir command|NA"
if {$debug} {debugw "[gts] $module timeout after $prompt_timeout seconds waiting for command prompt from $hname after entering dir command - exit 8"}
exit 8
}
}
#------------------------------------------------
# loop through list of files - put each file to remote sftp server
#------------------------------------------------
set hfile [open $lfile r] ; set data [read $hfile] ; close $hfile
set flist [string map {"\n" " "} "$data" ] ; set flist [string trim "$flist"]
set flist [split $flist " "] ; set nfiles [llength $flist]
if {$debug} {debugw "[gts] $module $nfiles file names found in list file $lfile"}
set ct 0
while {$ct < $nfiles} {
set nextfile [lindex $flist $ct]
if {$debug} {debugw "[gts] $module starting SFTP Put for filename $nextfile"}
set dct [expr $ct + 1]
set timeout $sftp_put_timeout
send "put $nextfile\r"
expect {
eof {
set response $expect_out(0,string)
echo "sftp EOF waiting for command prompt after entering put command|$nextfile"
if {$debug} {debugw "[gts] $module put file $dct - $nextfile failed"}
if {$debug} {debugw "[gts] $module EOF response from Put:\n$response"}
if {$debug} {debugw "[gts] $module EOF received waiting for prompt from $hname after entering put command - exit 8"}
exit 8
}
"100%*$sftp_prompt" {
if {$debug} {debugw "[gts] $module received PUT 100% and command prompt after PUT"}
if {$debug} {debugw "[gts] $module put file $dct - $nextfile succeeded"}
}
timeout {
echo "sftp timeout after $sftp_put_timeout seconds waiting for prompt after entering put command|$nextfile"
if {$debug} {debugw "[gts] $module timeout after $sftp_put_timeout seconds waiting for command prompt from $hname after entering put command - exit 8"}
exit 8
}
}
incr ct
}
#--------------------------------------------------------------
# post transfer directory list
#--------------------------------------------------------------
set timeout $prompt_timeout
send "ls -l\r"
expect {
eof {
echo "sftp EOF waiting for command prompt after entering dir command|NA"
if {$debug} {debugw "[gts] $module EOF received waiting for prompt from $hname after entering dir command - exit 8"}
exit 8
}
"$sftp_prompt" {
set response $expect_out(0,string)
set dir [lrange [split "$response" "\r"] 1 end-1]
set dir [string map {\{ "" \} ""} "$dir"]
set dir [string trim "$dir"]
if {$debug} {debugw "[gts] $module dir: \r$dir"}
if {$debug} {debugw "[gts] $module received command prompt after dir; will execute QUIT command"}
}
timeout {
echo "sftp timeout after $prompt_timeout seconds waiting for prompt after entering dir command|NA"
if {$debug} {debugw "[gts] $module timeout after $prompt_timeout seconds waiting for command prompt from $hname after entering dir command - exit 8"}
exit 8
}
}
#---------------------------------------------------------
# sftp checks for spawn EOF and will terminate prematurely
# unless you add extra expect command
#---------------------------------------------------------
set timeout $prompt_timeout
send "quit\r"
expect {
eof {
if {$debug} {debugw "[gts] $module EOF expected and received after QUIT command|NA"}
exit 0
}
"$sftp_prompt" {
echo "unexpected sftp prompt after entering QUIT command - should be EOF|NA"
if {$debug} {debugw "[gts] $module unexpected sftp prompt after entering QUIT command - should be EOF - exit 2"}
exit 2
}
timeout {
echo "sftp timeout after $prompt_timeout seconds waiting for EOF after QUIT command|NA"
if {$debug} {debugw "[gts] $module sftp timeout after $prompt_timeout seconds waiting for EOF after QUIT command - exit 8"}
exit 8
}
}
exit 0