Re: [arch-general] Syncing the mirrors
On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee <dpmcgee@gmail.com> wrote On Mon, Mar 8, 2010 at 7:09 PM, keenerd <keenerd@gmail.com> wrote:
There does not seem to be a documented "standard rsync" command for the mirrors to use, so I'm making all sorts of wild assumptions about what a mirror's rsync is doing. http://wiki.archlinux.org/index.php/Local_Mirror#The_synchronization_script
The real mirrors use that too? The whole "local mirror" title kind of threw me. I was looking at http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors . My bad. -Kyle
On Mon, Mar 8, 2010 at 7:20 PM, keenerd <keenerd@gmail.com> wrote:
On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee <dpmcgee@gmail.com> wrote On Mon, Mar 8, 2010 at 7:09 PM, keenerd <keenerd@gmail.com> wrote:
There does not seem to be a documented "standard rsync" command for the mirrors to use, so I'm making all sorts of wild assumptions about what a mirror's rsync is doing. http://wiki.archlinux.org/index.php/Local_Mirror#The_synchronization_script
The real mirrors use that too? The whole "local mirror" title kind of threw me. I was looking at http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors . My bad.
I've heard people be pointed in that direction in the past, so I would guess it is that or something similar. -Dan
On 09.03.2010 02:20, keenerd wrote:
On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee <dpmcgee@gmail.com> wrote On Mon, Mar 8, 2010 at 7:09 PM, keenerd <keenerd@gmail.com> wrote:
There does not seem to be a documented "standard rsync" command for the mirrors to use, so I'm making all sorts of wild assumptions about what a mirror's rsync is doing. http://wiki.archlinux.org/index.php/Local_Mirror#The_synchronization_script
The real mirrors use that too? The whole "local mirror" title kind of threw me. I was looking at http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors . My bad.
Afaik not every mirror uses that script and it's quite hard to reach all the admins because there is no mailinglist (though I've heared it's going to be set up). -- Florian Pritz -- {flo,bluewind}@server-speed.net
On 03/09/2010 03:17 PM, Florian Pritz wrote:
On 09.03.2010 02:20, keenerd wrote:
On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee<dpmcgee@gmail.com> wrote On Mon, Mar 8, 2010 at 7:09 PM, keenerd<keenerd@gmail.com> wrote:
There does not seem to be a documented "standard rsync" command for the mirrors to use, so I'm making all sorts of wild assumptions about what a mirror's rsync is doing. http://wiki.archlinux.org/index.php/Local_Mirror#The_synchronization_script
The real mirrors use that too? The whole "local mirror" title kind of threw me. I was looking at http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors . My bad.
Afaik not every mirror uses that script and it's quite hard to reach all the admins because there is no mailinglist (though I've heared it's going to be set up).
are you taking from your experience or you did find a mirror that does store old packages? afaik only kernel.org mirror does that but they were contacted. -- Ionut
On 09.03.2010 14:37, Ionut Biru wrote:
On 03/09/2010 03:17 PM, Florian Pritz wrote:
On 09.03.2010 02:20, keenerd wrote:
On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee<dpmcgee@gmail.com> wrote On Mon, Mar 8, 2010 at 7:09 PM, keenerd<keenerd@gmail.com> wrote:
There does not seem to be a documented "standard rsync" command for the mirrors to use, so I'm making all sorts of wild assumptions about what a mirror's rsync is doing. http://wiki.archlinux.org/index.php/Local_Mirror#The_synchronization_script
The real mirrors use that too? The whole "local mirror" title kind of threw me. I was looking at http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors . My bad.
Afaik not every mirror uses that script and it's quite hard to reach all the admins because there is no mailinglist (though I've heared it's going to be set up).
are you taking from your experience or you did find a mirror that does store old packages? afaik only kernel.org mirror does that but they were contacted.
<Romashka> I'm trying to force all admins to use the same rsync options Though that's from February and I don't what's the current status. Multi tier mirroring is being done atm and I've edited the mirror script from the wiki to fetch only checksums and thus being able to run every minute which would greatly increase the distribution speed, but I still have to test for bugs. If anyone wants to help: http://karif.server-speed.net/~flo/tmp/mirrorsync.sh.txt -- Florian Pritz -- {flo,bluewind}@server-speed.net
On Tue, Mar 9, 2010 at 8:22 AM, Florian Pritz <bluewind@server-speed.net> wrote:
On 09.03.2010 14:37, Ionut Biru wrote:
On 03/09/2010 03:17 PM, Florian Pritz wrote:
On 09.03.2010 02:20, keenerd wrote:
On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee<dpmcgee@gmail.com> wrote On Mon, Mar 8, 2010 at 7:09 PM, keenerd<keenerd@gmail.com> wrote:
There does not seem to be a documented "standard rsync" command for the mirrors to use, so I'm making all sorts of wild assumptions about what a mirror's rsync is doing. http://wiki.archlinux.org/index.php/Local_Mirror#The_synchronization_script
The real mirrors use that too? The whole "local mirror" title kind of threw me. I was looking at http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors . My bad.
Afaik not every mirror uses that script and it's quite hard to reach all the admins because there is no mailinglist (though I've heared it's going to be set up).
are you taking from your experience or you did find a mirror that does store old packages? afaik only kernel.org mirror does that but they were contacted.
<Romashka> I'm trying to force all admins to use the same rsync options
Though that's from February and I don't what's the current status.
Multi tier mirroring is being done atm and I've edited the mirror script from the wiki to fetch only checksums and thus being able to run every minute which would greatly increase the distribution speed, but I still have to test for bugs. If anyone wants to help: http://karif.server-speed.net/~flo/tmp/mirrorsync.sh.txt
We're bandwidth and rsync-slot constrained, not "slow at distribution". It seems like this solution is not solving a problem we currently have... -Dan
On 09.03.2010 15:25, Dan McGee wrote:
We're bandwidth and rsync-slot constrained, not "slow at distribution". It seems like this solution is not solving a problem we currently have...
Actually that's being solved by multi tier mirroring and the more tiers there are the slower the whole process is going to be so we're going to have that problem soon enough. -- Florian Pritz -- {flo,bluewind}@server-speed.net
participants (4)
-
Dan McGee
-
Florian Pritz
-
Ionut Biru
-
keenerd