NAME
SYNOPSIS
DESCRIPTION
OPTIONS
EXIT STATUS
FILES
NOTES
BUGS
SEE ALSO
r2sync - synchronize two (possibly remote) files or directories
r2sync [ options ] target1 target2
r2sync [ options ] profile
r2sync -d [ -p port ]
r2sync
synchronizes two files or two directories (the
“targets”). One or both may be on remote
machines. The targets can be specified in one of four ways:
relative/path/to/dir
/absolute/path/to/dir
The target can specify a path on the local machine. The path may be absolute (starting with a slash) or relative. E.g.,
r2sync Images/2018 backups/now
synchronizes two relative paths.
host:relative/path
host:/absolute/path
user@host:relative/path
user@host:/absolute/path
The target may be prefixed with the name of a remote machine and a colon (:) to indicate a path on that host. The path may be absolute or relative. If it is relative, it is relative to the home directory of the user on the remote host. E.g.,
r2sync Images/2018 bb.example.com:backups/now
synchronizes between a local path and a path relative to the user’s home directory on “bb.example.com”. r2sync uses ssh to log in to the remote host.
If the user name on the remote machine is not the same as on the local machine, add the name before the host name, e.g., “john@bb.example.com”. (The remote user name can also be specified in the ssh configuration file, see ssh_config(5).)
The remote machine must have a version of r2sync installed and in the user’s PATH.
r2sync://host/absolute/path
r2sync://host:port/absolute/path
r2sync can also communicate with already running r2sync servers, instead of starting them itself. (Such servers can be started with the -d option, see below.) In this case, the target is specified in the form of a URL starting with “r2sync://”. E.g.,
r2sync Images/2018 r2sync://example.com/home/store
In this case the path is always absolute (“/home/store” in the example). r2sync connects to port 874 on the indicated host. If the server is listening on another port, the port can be specified explicitly, e.g., “r2sync://bb.example.com:7348/”.
Instead of a port number, the URL may also contain the name of a well-known service (as found in /etc/services). E.g., if an r2sync server happens to be running on port 23, the telnet port, the URL may be written as
r2sync Images/2018 r2sync://example.com:telnet/home/store
profile
Instead of giving the two targets on the command line, the targets can also be read from a profile. The profile is a simple name, which corresponds to a file with extension .prf in ~/.r2sync (i.e., the directory .r2sync in the user’s home directory). E.g.,
r2sync my-music
reads the targets from the file ~/.r2sync/my-music.prf. See under FILES below for the format of profile files.
The state of the files or directories after the synchronization is stored in a file with extension .store in the directory ~/.r2sync. When r2sync is run again with the same targets, this stored state allows it to detect which files have changed since the previous synchronization. Each store is specific to a target pair.
r2sync uses the rsync algorithm to update files, which means that it detects which parts of a file are different compared to the same file on the other target and only sends enough bytes to patch the file. More precisely, it divides a file that needs to be updated in blocks, computes checksums for those blocks and then detects blocks with the same checksum in the file on the other target. Identical blocks can be detected even if they are in different positions or in a different order. This allows r2sync to synchronize large files and large directories over slow networks, because it transfers not much more than the bytes that actually changed.
Interactive
mode
r2sync normally runs in interactive mode, which means
that it presents one by one all the files that have changed,
are new, or have been deleted on one of the targets and then
asks the user what to do with them: copy from target1 to
target2 or the other way round, or skip the file. But it is
also possible to run r2sync in batch mode, with the
option -b. In that case all the files that only
changed on one target are automatically copied to the other
target and files that changed on both sides are skipped
(left as they are).
In interactive mode, r2sync shows the name of a file and indicates for both targets whether the file is unchanged, updated, new or deleted. The user then presses a key to indicate the action to take: > to copy the file from target1 to target2, < to copy it from target2 to target1, or / to skip the file. If there is a default action indicated, the user may also press the enter key to accept the default action.
A few other key presses are also possible: x asks r2sync to display the type of file, the permission bits, the modification time and the size of the file on both targets. The key q (for “quit”) causes r2sync to stop immediately without doing any synchronization. ? displays a short help text about the possible key presses. p (for “previous”) goes back to the previous file and asks to chose an action for that file again. g (“go”) starts synchronizing the files immediately, using the default actions for the remaining files. (In other words: it switches to batch mode.) And G also starts synchronizing the files immediately, but skips all remaining files.
After the last file, if batch mode has not been turned on, r2sync asks for confirmation before it starts to synchronize the files.
Progress
display
When r2sync starts, it forks two daemon processes,
one for each target (unless the the target is a URL, in
which case r2sync connects to an existing server for
that target). The daemons are responsible for reading,
writing and updating the files and the store at each target,
while the master process acts as a manager, interacting with
the user and issuing instructions to each daemon.
Unless it runs in quiet mode (-q), r2sync shows the progress of the synchronization. It prints a line with three parts, e.g.,:
-> pictures/holidays-1968/4567.jpg 5 min ETA
The first part is either “<-” or “->”, meaning, respectively, that r2sync is copying a file from target2 to target1, or from target1 to target2. The second part is the name of the file being synchronized. If the name is too long, some of it may be omitted. The third part is the expected time remaining to synchronize all the files, in seconds, minutes or hours.
When a file has been synchronized, the first part changes into “<=”, “=>”, “<*” or “*>”. The * means that r2sync was able to take a shortcut: It found a local file that was identical to the updated file and copied it.
Three-way
sync
If you have three targets that are synchronized, say
A, B and C, and you change some files
on A, you can update B and C with two
calls to r2sync:
r2sync A B
r2sync A C
Now in theory B and C are also synchronized, but r2sync does not know that yet. The fact that the files have the same sizes and the same modification times on both B and C is not enough to conclude that the contents are the same. If you call
r2sync B C
it will report that some files changed on both sides and will ask what to do. If you are sure the only changes on B and C are due to synchronizing with A, one solution is to use the -c (--conflict-resolution) option to set the default action:
r2sync -c ’>’ B C
and then accept that action on all updated files with the g command. Or, to avoid being prompted, use batch mode:
r2sync -b -c ’>’ B C
A safer option is to use the -a option (--autocheck),
r2sync -a B C
which causes r2sync to compute checksums for the conflicting files to see if they are actually the same. If they are the same, there is no conflict and no action is required. Computing the checksums can take some time, though, depending on the number of files that changed on both sides and the speed of the computer.
r2sync
accepts both short and long options. Long and short options
can be mixed. Long options can be abbreviated to a unique
prefix, e.g., --con instead of
--conflict-resolution. Short options can be combined,
e.g., -bs instead of -b -s. Arguments to short
options can either be written directly after the option
(-p1500) or separated by whitespace (-p 1500).
Arguments to long options can be separated by whitespace
(--port 1500) or by an equals sign
(--port=1500).
-b, --batch
Run in batch mode. In batch mode, r2sync does not ask what actions to apply to each pair of new or updated files, but automatically executes the default action. (See also -c to set the default action for conflicts.)
-k, --keep-going
Tells r2sync to continue after some errors. Normally, when an error occurs while synchronizing a file, r2sync prints an error and exits. With this option, r2sync prints an error but then continues with the next file. -k does not change the exit code of r2sync, which will still be non-zero if any error occurred.
-C, --compression
If one or both of the targets are on remote machines, this option is passed on to the ssh command. It requests ssh to compress the communication. This may speed up a synchronization between fast machines on a slow network, but may slow it down between slow machines on a fast network.
-d, --daemon
Run in daemon mode. In this mode, r2sync implements a server that accepts commands on the standard input according to the r2sync protocol. (With the -p option, r2sync listens on a socket instead of standard input.) This option is used by r2sync when is starts a server on a remote machine to communicate with. The -d option can not be combined with any other option, except the -p option.
-e pattern, --exclude pattern
Specifies a path to skip, i.e., not synchronize. This option can be used multiple times. The pattern is matched after any patterns already in the profile., I.e., it can override those patterns. (See FILES below for the syntax of the profile file.)
The pattern may include wildcards *, ? and character classes. The * matches zero or more characters. The ? matches exactly one character. A character class is a set of characters between [ and ]. It matches a single character from that set. If the first character after the [ is a !, it instead matched a single character that is not in the set. The set may also contain ranges, e.g., [ABC0-9] matches an A, a B, a C or any of the digits 0 to 9. To avoid that a dash (-) is interpreted as a range, make it the first or last character in the set.
Unlike in the shell, wildcards and character classes can match slashes and initial dots, too. E.g., a*b will match a path “alpha/.local/job”. See under FILES below for more examples.
Note that the wildcards have special meaning in the shell and should be escaped or quoted on the command line:
r2sync -e
’*.old’ ...
r2sync -e old ...
-i pattern, --include pattern
Specifies a path that should not be skipped. This option can be used multiple times. The pattern has the same syntax as for the - e option. The order of -e and -i is significant. If a file matches multiple patterns, the pattern specified last wins. E.g., with the options -e *.c -i a* the file “alpha.c” matches both patterns, but the last pattern it matches is an inclusion and thus the file will be included in the synchronization.
-p port, --port port
Listen on a network socket instead of reading and writing standard input and output. The port is a number or a well-known service name. See /etc/services for the service names. This option may only be used if -d. is also specified. Note that unprivileged users cannot start daemons that listen on ports below 1024.
-P n, --protocol n
Set the protocol that r2sync should use to communicate with its two daemons. The protocol is a number greater than zero and it must not be higher than the highest protocol r2sync understands. If r2sync communicates with already running daemons or with a remote daemon of r2sync, the actual protocol used may be lower, if those daemons only understand older protocols.
Try running r2sync -v to see what the highest protocol is that r2sync supports.
-c action, --conflict-resolution action
Specify what to do with files that changed (or are new) on both targets. This can be set to <, > or /. < sets the default to copying from target 2 to target 1. > is the opposite. And / (the default) is to skip the files. In interactive mode, this option sets the default and the user still has to press Enter to select the action (or another key to choose a different action). In batch mode (-b), the default action is applied automatically.
Note that < and > have special meaning to the shell, so they must be quoted on the command line: r2sync -c ’>’ ...
-n, --noshortcuts
Don’t look for identical files that can be copied locally (shortcuts). Normally, when a file is about to be updated, r2sync computes a strong checksum over the up-to-date file so the daemon with the outdated file can check if it already received another file with the same checksum. If so, it will not use the rsync algorithm, but copy the earlier file, avoiding most network traffic. However, on very slow computers and if there are many files and few duplicates, computing checksums may take more time than copying the occasional duplicate file and therefore this feature can be disabled.
Option -n cannot be used together with -a.
-a,--autocheck
Normally, when r2sync detects a conflict, i.e., a file that changed or is new on both targets, it asks the user what to do, or, in batch mode, applies the action set with -c. The option -a causes r2sync to compare such files to see if they aren’t actually the same. If they are, the user does not neeed to be asked and no action needs to be applied to the files. Checking if the files are the same involves comparing the last modification time, the file size and the checksums. Because computing the checksums can take some time, this is not the default.
Option -a cannot be used together with -n.
-q, --quiet
Don’t show any progress messages.
-r, --reset-logs
If there is a log of the previous synchronization between the two targets, make it empty. This has the effect that all files in the two targets will be considered to be new files.
-s, --statistics
Print some statistics at the end of the synchronization about the communication between the master process and the two targets: the number of files reported by each target, their total size and the number of bytes received from and sent to each target. The numbers only count the bytes processed by r2sync. They do not take into account the effect of any compression or overhead due to ssh(1).
-v, --version
Print the program’s version number and highest supported protocol version, then exit.
-h, --help
Show a short usage message and exit.
r2sync exits with a status of 0, unless an error occurred.
~/.r2sync/*.prf
The profile files contain exactly two targets to synchronize and zero or more lines to set options. Here is an example:
root =
/home/bert/Pictures
root = bert@luna.example.org:Pictures
exclude = *~
exclude = */.*
exclude = .*
quiet = yes
Empty lines and lines that start with # are ignored. Other lines must have a keyword and a value, separated by at least one space or equals sign. The currently recognized keywords are:
|
root |
Specifies the target, using the same syntax as on the command line. There must be exactly two lines defining a root. |
exclude
include
Specifies a path to skip, respectively not skip. See the -e option for the syntax. The order is significant, e.g.,
exclude a*
include ab*
exclude abc*
specifies that all paths that start with an “a” are to be ignored, unless they start with “ab”. But paths that start with “abc” are to be ignored nevertheless. So a path “anno” is ignored, because the last pattern it matches is “a*”, but “abon” is included, because the last pattern it matches is “ab*”.
Patterns specified on the command line with -e and -i are treated as if they were added after the patterns in the profile file.
compression
|
batch |
||
|
quiet |
statistics
autocheck
These correspond to the command line options of the same name. The value must be yes, true or 1 to turn the option on, or no, false or 0 to turn it off (the default). The command line options (-C, -b, -q and -s) override these setting, i.e., -C turns compression on, even if the profile contains
compression = no
conflict-resolution
Corresponds to -c. The value must be <, > or /. The command line option overrides this setting.
protocol
Corresponds to -P. The value must be a number. The command line option overrides this setting.
~/.r2sync/*.store
The .store files in ~/.r2sync log the results of previous synchronizations, so that r2sync can detect which files have changed since. They are simple text files. The first line contains “RSYNC”, a space, and the version number of the store format (currently 1). The next two lines are the two targets that were synchronized. And the rest of the lines contains the paths to the files that were synchronized (one per line), preceded by the file mode in octal, the modification time in decimal in seconds since the epoch and the file size in bytes in decimal. E.g.:
R2SYNC 1
/tmp/test
bert@nyx.example.org:/tmp/test
100644 1519311362 1744 file-a
100644 1519311362 864 file-b
The first version of r2sync was created by Bert Bos in February 2018. It was published on November 4, 2019. r2sync was inspired by unison(1) and rsync(1). It is meant as a replacement for unison, although it currently has far fewer options. What it does have, however, is a documented, text-based protocol between the master process and the servers. This should make the protocol independent of the version of r2sync itself and should allow other programs to implement the protocol, too. It is the intention that later versions of r2sync, even if they have new features, continue to support earlier versions of the protocol (as long as the new features aren’t used).
Connections to r2sync running as a server (with options -d and -p) are currently not encrypted and not authenticated. It is better to avoid -p unless you have taken measures to block untrusted clients from connecting.
There is currently no support for creating or modifying profiles. They must be edited by hand.
r2sync shows an estimate of the remaining time, but the estimate is very inaccurate. It is based on the size of the files already synchronized and the size of the files yet to synchronize, on the assumption that all files changed in similar ways. If all the changes are in the last files to synchronize, r2sync will underestimate the time; if all the changes are in the first files, it will overestimate the time. Also, it assumes that the network is equally fast in both directions. (Which isn’t true for an ADSL line, for example.)
For copying a file that doesn’t exist yet, r2sync is much slower than a direct copy, because the data is base64-encoded and passes through r2sync’s text-based protocol twice (from the source server to the master process and from the master process to the destination server).
r2sync relies on file modification times and file sizes to detect a changed file. If a file has been changed without altering its size and its modification time has been reset (e.g., with touch(1) or jhead(1) with option -ft), r2sync will not detect it. On the other hand, if a file’s modification time is set to a time before the last synchronization, that is detected.
Also, r2sync requires that the clocks of two machines to synchronize are sufficiently close. The chance is small, but if one machine’s clock is ahead, it may happen that a file on the other machine is updated and set to the time of the machine that is ahead, and that a subsequent edit of that file ends up not changing the modification time. If the edit also not changed the size, the change will go undetected.
Because a target machine may be addressed in various ways (host name, full domain name, IP address), r2sync does not rely on the address to identify a target, but on the machine-id(5), which is 32-character hexadecimal string found in /etc/machine-id on systems that use systemd(1). If that file does not exists, r2sync generates a random string itself on first use and stores it in ~/.r2sync/machine-id. If that string is changed, r2sync will not recognize the machine anymore and will assume all files are new.
Probably plenty more.
unison(1),
rsync(1), ssh(1), ssh_config(5),
tsocks(1), machine-id(5).
The rsync
algorithm.
The r2sync
protocol version 2.