Rsync --dry-run 未显示正确的发送字节

Rsync --dry-run 未显示正确的发送字节

我正在尝试使用rsyn--dry-run选项来查看是否有足够的空间用于真正的同步。出于测试目的,我尝试同步一个目录Documents。目录的大小是

  x@x:~$ du Documents
  ...
  640760    Documents/

我尝试与之同步的文件容器的大小是

  x@x:~$ df /media/veracrypt2
  Filesystem             1K-blocks  Used Available Use% Mounted on
  /dev/mapper/veracrypt2      9928  1191      8737  12% /media/veracrypt2

然后我运行rsync命令:

x@x:~$ rsync -ar --dry-run --stats Documents/ /media/veracrypt2

Number of files: 665 (reg: 560, dir: 105)
Number of created files: 664 (reg: 560, dir: 104)
Number of deleted files: 0
Number of regular files transferred: 560
Total file size: 649,731,108 bytes
Total transferred file size: 649,731,108 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 19,429
Total bytes received: 2,115

sent 19,429 bytes  received 2,115 bytes  43,088.00 bytes/sec
total size is 649,731,108  speedup is 30,158.33 (DRY RUN)

我不明白为什么只发送了19,429字节?容器是空的,所以该目录的所有文件都Documents应该被传输,意思是 649,731,108!?

我也尝试过使用较小的目录,Scripts其大小为

 du -h Scripts/
 32K    Scripts/test/Logs
 56K    Scripts/test
 116K   Scripts/Logs
 264K   Scripts/color_schemes
 580K   Scripts/

这里应该能够复制整个目录。当运行rsync该目录时,我得到输出:

sending incremental file list
./
after_install.sh
install-crafter.sh
install-eclipse.sh
mk_autostart_app.sh
package_backup.sh
pandora.sh
sync_script.sh
trackpoint_speed_sens.sh
wallpaper.sh
Logs/
Logs/LOG_SYNC.log
Logs/LOG_SYNC.log~
Logs/LOG_WALLPAPER.txt
Logs/Log_sync.log
Logs/PANDORA.log
Logs/test
color_schemes/
color_schemes/kile.kateschema
test/
test/sync_script.sh
test/Logs/
test/Logs/Log_sync.log

Number of files: 25 (reg: 20, dir: 5)
Number of created files: 22 (reg: 18, dir: 4)
Number of deleted files: 0
Number of regular files transferred: 18
Total file size: 360,658 bytes
Total transferred file size: 353,166 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 900
Total bytes received: 95

sent 900 bytes  received 95 bytes  1,990.00 bytes/sec
total size is 360,658  speedup is 362.47 (DRY RUN)

那么为什么这里发送的数据只有928字节呢?该值不应该表明需要将多少字节复制到目的地吗?

答案1

--dry-run模式下,rsync确定需要传输哪些文件,但实际上并不传输其数据 - 因为自然没有必要这样做。这意味着它不执行增量匹配 - 因为它是数据传输逻辑的一部分。

数字反映这一点的原因是......嗯......代码显示实际的统计数据,而不是一些“可能的”统计数据。

以下是源树中的文件夹rsync -avvv --log-file=rsync.log --no-whole-file --stats doc doc2与正常同步之间的差异(带有剥离的时间戳、PID 差异和堆统计信息):docrsync--dry-run

 delta-transmission enabled
 recv_generator(doc,1)
 recv_generator(doc,2)
+set modtime of doc to (1463404939) Mon May 16 16:22:19 2016
 recv_generator(doc/README-SGML,3)
 recv_generator(doc/profile.txt,4)
 recv_generator(doc/rsync.sgml,5)
 send_files(2, doc)
 cd+++++++++ doc/
 send_files(3, doc/README-SGML)
+send_files mapped doc/README-SGML of size 672
+calling match_sums doc/README-SGML
+sending file_sum
+false_alarms=0 hash_hits=0 matches=0
+>f+++++++++ doc/README-SGML
+sender finished doc/README-SGML
 send_files(4, doc/profile.txt)
+send_files mapped doc/profile.txt of size 1935
+calling match_sums doc/profile.txt
+sending file_sum
+false_alarms=0 hash_hits=0 matches=0
+>f+++++++++ doc/profile.txt
+sender finished doc/profile.txt
 send_files(5, doc/rsync.sgml)
+send_files mapped doc/rsync.sgml of size 11843
+calling match_sums doc/rsync.sgml
+sending file_sum
+false_alarms=0 hash_hits=0 matches=0
+>f+++++++++ doc/rsync.sgml
+sender finished doc/rsync.sgml
 recv_files(1) starting
 recv_files(doc)
 recv_files(doc/README-SGML)
+got file_sum
+set modtime of doc/.README-SGML.hkH0u5 to (1463404939) Mon May 16 16:22:19 2016
+renaming doc/.README-SGML.hkH0u5 to doc/README-SGML
 recv_files(doc/profile.txt)
+got file_sum
+set modtime of doc/.profile.txt.Wdf4x9 to (1463404939) Mon May 16 16:22:19 2016
+renaming doc/.profile.txt.Wdf4x9 to doc/profile.txt
 recv_files(doc/rsync.sgml)
+got file_sum
+set modtime of doc/.rsync.sgml.JSte5H to (1463404939) Mon May 16 16:22:19 2016
+renaming doc/.rsync.sgml.JSte5H to doc/rsync.sgml
 generate_files phase=1
+set modtime of doc to (1463404939) Mon May 16 16:22:19 2016
 send_files phase=1
 recv_files phase=1
 generate_files phase=2
 send_files phase=2
 send files finished
-total: matches=0  hash_hits=0  false_alarms=0 data=0
+total: matches=0  hash_hits=0  false_alarms=0 data=14450
 Number of files: 4 (reg: 3, dir: 1)
 Number of created files: 4 (reg: 3, dir: 1)
 Number of deleted files: 0
 Number of regular files transferred: 3
 Total file size: 14,450 bytes
 Total transferred file size: 14,450 bytes
-Literal data: 0 bytes
+Literal data: 14,450 bytes
 Matched data: 0 bytes
 File list size: 0
 File list generation time: 0.001 seconds
 File list transfer time: 0.000 seconds
-Total bytes sent: 153
-Total bytes received: 793
-sent 153 bytes  received 793 bytes  378.40 bytes/sec
-total size is 14,450  speedup is 15.27 (DRY RUN)
-[sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0) (DRY RUN)
+Total bytes sent: 14,723
+Total bytes received: 1,435
+sent 14,723 bytes  received 1,435 bytes  4,616.57 bytes/sec
+total size is 14,450  speedup is 0.89
+[sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0)

以下是初始试运行和我 1) 进行真正同步、2) 编辑一个文件后的试运行之间的差异rsync.sgml。它表明增量匹配不是在试运行中完成的:

 received 3 names
 recv_file_list done
 get_local_name count=4 doc2
-created directory doc2
 delta-transmission enabled
 recv_generator(doc,1)
 recv_generator(doc,2)
 recv_generator(doc/README-SGML,3)
+doc/README-SGML is uptodate
 recv_generator(doc/profile.txt,4)
+doc/profile.txt is uptodate
 recv_generator(doc/rsync.sgml,5)
 send_files(2, doc)
<...>
 Number of files: 4 (reg: 3, dir: 1)
-Number of created files: 4 (reg: 3, dir: 1)
+Number of created files: 0
 Number of deleted files: 0
-Number of regular files transferred: 3
-Total file size: 14,450 bytes
-Total transferred file size: 14,450 bytes
+Number of regular files transferred: 1
+Total file size: 14,476 bytes
+Total transferred file size: 11,869 bytes
 Literal data: 0 bytes
 Matched data: 0 bytes
 File list size: 0
-File list generation time: 0.010 seconds
+File list generation time: 0.001 seconds
 File list transfer time: 0.000 seconds
-Total bytes sent: 153
-Total bytes received: 793
-sent 153 bytes  received 793 bytes  1,892.00 bytes/sec
-total size is 14,450  speedup is 15.27 (DRY RUN)
+Total bytes sent: 157
+Total bytes received: 830
+sent 157 bytes  received 830 bytes  658.00 bytes/sec
+total size is 14,476  speedup is 14.67 (DRY RUN)
 [sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0) (DRY RUN)

现在,关于您的任务 - 检查目的地是否有足够的空间进行真正的同步。

从差异中可以看出,

  • 文件被一份一份地复制,
  • 每个文件都被写入一个临时名称,然后移动到原始文件上

因此,目的地所需的空间量为:

  • sum( max(existing_size,new_size) for all files to be synced) + max( (new_size) for all files to be synced)

第一项是任何时刻“最终”数据量的最坏情况,第二项是临时副本的空间。

每个文件大小可以填充到存储单元大小的倍数,以允许浪费空间(如果适用于目标 FS 并且预期总数量足够高以产生差异)。

相关内容