我正在尝试使用rsyn
该--dry-run
选项来查看是否有足够的空间用于真正的同步。出于测试目的,我尝试同步一个目录Documents
。目录的大小是
x@x:~$ du Documents
...
640760 Documents/
我尝试与之同步的文件容器的大小是
x@x:~$ df /media/veracrypt2
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/veracrypt2 9928 1191 8737 12% /media/veracrypt2
然后我运行rsync
命令:
x@x:~$ rsync -ar --dry-run --stats Documents/ /media/veracrypt2
Number of files: 665 (reg: 560, dir: 105)
Number of created files: 664 (reg: 560, dir: 104)
Number of deleted files: 0
Number of regular files transferred: 560
Total file size: 649,731,108 bytes
Total transferred file size: 649,731,108 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 19,429
Total bytes received: 2,115
sent 19,429 bytes received 2,115 bytes 43,088.00 bytes/sec
total size is 649,731,108 speedup is 30,158.33 (DRY RUN)
我不明白为什么只发送了19,429字节?容器是空的,所以该目录的所有文件都Documents
应该被传输,意思是 649,731,108!?
我也尝试过使用较小的目录,Scripts
其大小为
du -h Scripts/
32K Scripts/test/Logs
56K Scripts/test
116K Scripts/Logs
264K Scripts/color_schemes
580K Scripts/
这里应该能够复制整个目录。当运行rsync
该目录时,我得到输出:
sending incremental file list
./
after_install.sh
install-crafter.sh
install-eclipse.sh
mk_autostart_app.sh
package_backup.sh
pandora.sh
sync_script.sh
trackpoint_speed_sens.sh
wallpaper.sh
Logs/
Logs/LOG_SYNC.log
Logs/LOG_SYNC.log~
Logs/LOG_WALLPAPER.txt
Logs/Log_sync.log
Logs/PANDORA.log
Logs/test
color_schemes/
color_schemes/kile.kateschema
test/
test/sync_script.sh
test/Logs/
test/Logs/Log_sync.log
Number of files: 25 (reg: 20, dir: 5)
Number of created files: 22 (reg: 18, dir: 4)
Number of deleted files: 0
Number of regular files transferred: 18
Total file size: 360,658 bytes
Total transferred file size: 353,166 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 900
Total bytes received: 95
sent 900 bytes received 95 bytes 1,990.00 bytes/sec
total size is 360,658 speedup is 362.47 (DRY RUN)
那么为什么这里发送的数据只有928字节呢?该值不应该表明需要将多少字节复制到目的地吗?
答案1
在--dry-run
模式下,rsync
确定需要传输哪些文件,但实际上并不传输其数据 - 因为自然没有必要这样做。这意味着它不执行增量匹配 - 因为它是数据传输逻辑的一部分。
数字反映这一点的原因是......嗯......代码显示实际的统计数据,而不是一些“可能的”统计数据。
以下是源树中的文件夹rsync -avvv --log-file=rsync.log --no-whole-file --stats doc doc2
与正常同步之间的差异(带有剥离的时间戳、PID 差异和堆统计信息):doc
rsync
--dry-run
delta-transmission enabled
recv_generator(doc,1)
recv_generator(doc,2)
+set modtime of doc to (1463404939) Mon May 16 16:22:19 2016
recv_generator(doc/README-SGML,3)
recv_generator(doc/profile.txt,4)
recv_generator(doc/rsync.sgml,5)
send_files(2, doc)
cd+++++++++ doc/
send_files(3, doc/README-SGML)
+send_files mapped doc/README-SGML of size 672
+calling match_sums doc/README-SGML
+sending file_sum
+false_alarms=0 hash_hits=0 matches=0
+>f+++++++++ doc/README-SGML
+sender finished doc/README-SGML
send_files(4, doc/profile.txt)
+send_files mapped doc/profile.txt of size 1935
+calling match_sums doc/profile.txt
+sending file_sum
+false_alarms=0 hash_hits=0 matches=0
+>f+++++++++ doc/profile.txt
+sender finished doc/profile.txt
send_files(5, doc/rsync.sgml)
+send_files mapped doc/rsync.sgml of size 11843
+calling match_sums doc/rsync.sgml
+sending file_sum
+false_alarms=0 hash_hits=0 matches=0
+>f+++++++++ doc/rsync.sgml
+sender finished doc/rsync.sgml
recv_files(1) starting
recv_files(doc)
recv_files(doc/README-SGML)
+got file_sum
+set modtime of doc/.README-SGML.hkH0u5 to (1463404939) Mon May 16 16:22:19 2016
+renaming doc/.README-SGML.hkH0u5 to doc/README-SGML
recv_files(doc/profile.txt)
+got file_sum
+set modtime of doc/.profile.txt.Wdf4x9 to (1463404939) Mon May 16 16:22:19 2016
+renaming doc/.profile.txt.Wdf4x9 to doc/profile.txt
recv_files(doc/rsync.sgml)
+got file_sum
+set modtime of doc/.rsync.sgml.JSte5H to (1463404939) Mon May 16 16:22:19 2016
+renaming doc/.rsync.sgml.JSte5H to doc/rsync.sgml
generate_files phase=1
+set modtime of doc to (1463404939) Mon May 16 16:22:19 2016
send_files phase=1
recv_files phase=1
generate_files phase=2
send_files phase=2
send files finished
-total: matches=0 hash_hits=0 false_alarms=0 data=0
+total: matches=0 hash_hits=0 false_alarms=0 data=14450
Number of files: 4 (reg: 3, dir: 1)
Number of created files: 4 (reg: 3, dir: 1)
Number of deleted files: 0
Number of regular files transferred: 3
Total file size: 14,450 bytes
Total transferred file size: 14,450 bytes
-Literal data: 0 bytes
+Literal data: 14,450 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
-Total bytes sent: 153
-Total bytes received: 793
-sent 153 bytes received 793 bytes 378.40 bytes/sec
-total size is 14,450 speedup is 15.27 (DRY RUN)
-[sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0) (DRY RUN)
+Total bytes sent: 14,723
+Total bytes received: 1,435
+sent 14,723 bytes received 1,435 bytes 4,616.57 bytes/sec
+total size is 14,450 speedup is 0.89
+[sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0)
以下是初始试运行和我 1) 进行真正同步、2) 编辑一个文件后的试运行之间的差异rsync.sgml
。它表明增量匹配不是在试运行中完成的:
received 3 names
recv_file_list done
get_local_name count=4 doc2
-created directory doc2
delta-transmission enabled
recv_generator(doc,1)
recv_generator(doc,2)
recv_generator(doc/README-SGML,3)
+doc/README-SGML is uptodate
recv_generator(doc/profile.txt,4)
+doc/profile.txt is uptodate
recv_generator(doc/rsync.sgml,5)
send_files(2, doc)
<...>
Number of files: 4 (reg: 3, dir: 1)
-Number of created files: 4 (reg: 3, dir: 1)
+Number of created files: 0
Number of deleted files: 0
-Number of regular files transferred: 3
-Total file size: 14,450 bytes
-Total transferred file size: 14,450 bytes
+Number of regular files transferred: 1
+Total file size: 14,476 bytes
+Total transferred file size: 11,869 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
-File list generation time: 0.010 seconds
+File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
-Total bytes sent: 153
-Total bytes received: 793
-sent 153 bytes received 793 bytes 1,892.00 bytes/sec
-total size is 14,450 speedup is 15.27 (DRY RUN)
+Total bytes sent: 157
+Total bytes received: 830
+sent 157 bytes received 830 bytes 658.00 bytes/sec
+total size is 14,476 speedup is 14.67 (DRY RUN)
[sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0) (DRY RUN)
现在,关于您的任务 - 检查目的地是否有足够的空间进行真正的同步。
从差异中可以看出,
- 文件被一份一份地复制,
- 每个文件都被写入一个临时名称,然后移动到原始文件上
因此,目的地所需的空间量为:
sum( max(existing_size,new_size) for all files to be synced) + max( (new_size) for all files to be synced)
第一项是任何时刻“最终”数据量的最坏情况,第二项是临时副本的空间。
每个文件大小可以填充到存储单元大小的倍数,以允许浪费空间(如果适用于目标 FS 并且预期总数量足够高以产生差异)。