[arch-dev-public] Reducing traffic usage during package move between repos

Roman Kyrylych roman.kyrylych at gmail.com
Wed Sep 2 04:55:22 EDT 2009

On Wed, Sep 2, 2009 at 11:41, Dieter Plaetinck<dieter at plaetinck.be> wrote:
> I think a big improvemt will be when we fix it so that if we move packages from testing to extra/core, mirrors will not see as a "file remove and a new file", which causes a lot of unneeded traffic.
> if we make this system use symlinks or whatever this can be handled much better.
> but other then that, I think the current syncing system is fine. (the "getting" part is something different. see tiered setup,mirrorbrain etc. discussions)

Since we already use symlinks for -any packages this proposal looks okay.
The layout could be like this then:
<repo>/os/i686/foobar-1.2.3-1-i686.pkg.tar.gz ->
<repo>/os/x86_64/foobar-1.2.3-1-x86_64.pkg.tar.gz ->
<repo>/os/any/foobaz-4.5.6-1-any.pkg.tar.gz ->
<repo>/os/all/foobar-1.2.3-1-i686.pkg.tar.gz ->
<repo>/os/all/foobar-1.2.3-1-x86_64.pkg.tar.gz ->
<repo>/os/all/foobaz-4.5.6-1-any.pkg.tar.gz ->

As mentioned above - this will reduce the amount of traffic during
package moves between repos.
The negative side is that this will need one-time redownload of all
packages by mirrors,
though this can be avoided by manual moving of the files by mirror admin
(need to be done in this way: wait until /all syncing starts, kill rsync,
move packages manually, start rsync).

BTW I always wondered why we have that silly "os" subdir.
Would be nice to get rid of it during this big mirroring change (if
it'll happen).

Roman Kyrylych (Роман Кирилич)

More information about the arch-dev-public mailing list