Is rsync stateless?

Question

Let's say we do a backup (2TB and 500'000 files):

rsync -az /home/me/ [email protected]:/backup/home/me/   

Questions:

• Does it first download from destination the list of files (+ their hashes) that are already present on destination, to be able to compute what needs to be sent?

  If yes, how big can this download be for 500k files? I guess at least 100 bytes per file (filename + the hash), i.e. at least 50MB? Bigger (because files are split in smaller chunks)?

• Is rsync stateless? Which one of these is correct:

  1. when a sync has been successful, no database is kept locally nor on distant computer. If I redo another sync 1 hour later, all will be re-done from scratch (hashing, re-download of the list of files already present on destination, etc.)

  or

  2. when a sync has been successful, a database is kept both locally and on the distant server with hashes etc. so that next synchronizations will be faster (no longer necessary to re-hash everything)

  ?

Answer 1

Rsync is stateless. Each time you re-run it, it will re-evaluate the files at the source and destination to decide what to transfer.

However, note that without the -c (--checksum) option, rsync does not calculate checksums. It only compares sizes and modification times. And also owner, group, and mode if you're using -a. This means that generating the list does not have to open and read any of the files, only the directories. The filesystem work that your rsync command has to do is approximately the same as running find /home/me -ls.

You can use the --stats option to get some useful info about how much data is being transferred. This includes a "Total bytes sent" field. You can run rsync with -n --stats, and this field will tell you how big the file list is that has to be transferred. (There's also a "File list size" reported, but I'm only getting 0 reported for some reason.) Note that rsync does not send the full path for each file, so the file list may be significantly smaller than 100 bytes per file.

You may also be interested in How Rsync Works: A Practical Overview.