[arch-general] rsync for mirroring

[Piyush P Kurur]

Hi,

I am part of a group of people who run a mirror for some GNU/Linux
distros.  I have a question regarding how rsync works. The question is
prompted by the frequent overloading of rsync mirrors. 


If I am not mistaken the overall idea behind rsync is the following (I
am assuming that dest wants to update form src)

(1) dest computes a signature of the current snapshot of its file
system sends to src.

(2) src goes over the its current snapshot and calculates a delta
(changes) from the signature that dest sends.

(3) src then sends the delta to dest and dest updates.


Although this seems fine for a single source single destination
system, doesint it lead to overloading of src when there are multiple
dest, like in the case of mirroring, src has to compute the delta
which is a time consuming process.

Alternatively if the src can do the delta computation once and for all
then it will improve the efficiency a lot. One can imagine that a
destination mirror can keep track of a snapshot time when it last
synced and send the src just this. src keeps track of all the deltas,
somewhat like rdiff-backup and then sends the dest all the deltas that
will get it uptodate.

Regards

ppk