[arch-mirrors] Outdated mirrors
Is it just me or are most of the arch mirrors outdated/slow-syncing? I sync every hour and its always the same. When there is a KDE update or so all the mirrors get outdated. This was a problem with the old mirroring scheme, and to me it seems to be the same now. Is this a problem with few Tier-1 mirrors or is archlinux.org still overstrained when there are huge updates?
I am ready to set up rsync and allow 10-20 mirrors (start with 10 and see how it works with the bandwidth) in Europe, preferred Germany - if this can help us get our mirrors synced fast even with such updates.
I think a bleeding-edge linux distribution should definitely not have issues with slow syncs. When there is a security update or an update like KDE it should be out there as fast as possible, no matter which mirror you use.
On Thu, Jun 3, 2010 at 20:31, Mihael Pranjić mpranj@limun.org wrote:
Is it just me or are most of the arch mirrors outdated/slow-syncing? I sync every hour and its always the same. When there is a KDE update or so all the mirrors get outdated. This was a problem with the old mirroring scheme, and to me it seems to be the same now. Is this a problem with few Tier-1 mirrors or is archlinux.org still overstrained when there are huge updates?
Not all mirrors switched to multi-tier scheme yet, however these are limited in bandwidth compared to tier1 mirrors, so syncing from tier 1 mirror should not be slow. According to mirror status page [1] the delay for manyof mirrors is not that bad, however I did not check these numbers during major repo updates.
I am ready to set up rsync and allow 10-20 mirrors (start with 10 and see how it works with the bandwidth) in Europe, preferred Germany - if this can help us get our mirrors synced fast even with such updates.
We already have 2 tier1 mirrors in Germany and 1 in UK, so I don't know if there is a need for another one, so feedback from potential tier2 mirrors would be welcome.
I think a bleeding-edge linux distribution should definitely not have issues with slow syncs. When there is a security update or an update like KDE it should be out there as fast as possible, no matter which mirror you use.
Do you suggest to increase sync time to less than 1 hour? Perhaps we could try that.
To every mirror admin subscribed to this list: please voice your opinion on the issues raised in this mail. Since you are the guys that provide the service - it is important for us to provide you with possibilities to make it better.
[1] https://www.archlinux.de/?page=MirrorStatus;orderby=delay;sort=asc
On Sat, Jun 19, 2010 at 15:11, Roman Kyrylych roman.kyrylych@gmail.com wrote:
We already have 2 tier1 mirrors in Germany and 1 in UK, so I don't know if there is a need for another one, so feedback from potential tier2 mirrors would be welcome.
Correction: we have only one tier1 mirror in Germany.
Am 2010-06-19 14:11, schrieb Roman Kyrylych:
On Thu, Jun 3, 2010 at 20:31, Mihael Pranjić mpranj@limun.org wrote:
Is it just me or are most of the arch mirrors outdated/slow-syncing? I sync every hour and its always the same. When there is a KDE update or so all the mirrors get outdated. This was a problem with the old mirroring scheme, and to me it seems to be the same now. Is this a problem with few Tier-1 mirrors or is archlinux.org still overstrained when there are huge updates?
Not all mirrors switched to multi-tier scheme yet, however these are limited in bandwidth compared to tier1 mirrors, so syncing from tier 1 mirror should not be slow. According to mirror status page [1] the delay for manyof mirrors is not that bad, however I did not check these numbers during major repo updates.
I am ready to set up rsync and allow 10-20 mirrors (start with 10 and see how it works with the bandwidth) in Europe, preferred Germany - if this can help us get our mirrors synced fast even with such updates.
We already have 2 tier1 mirrors in Germany and 1 in UK, so I don't know if there is a need for another one, so feedback from potential tier2 mirrors would be welcome.
I think a bleeding-edge linux distribution should definitely not have issues with slow syncs. When there is a security update or an update like KDE it should be out there as fast as possible, no matter which mirror you use.
Do you suggest to increase sync time to less than 1 hour? Perhaps we could try that.
Yip, I think syncing more often could improve the situation, since it does not really make much more traffic. It's rather a problem if you sync only once a day so the main mirror has to deal with more traffic in less time. Syncing every 15 minutes (on a random minute) could even improve performance since we would sync 100mb now, 100 later, instead of having a dozen servers syncing 200mb at the same time. I could also be wrong.
However, another result of this (if applied to all mirrors) is that some probably don't sync from the tier-2 mirrors. Say server D syncs from C, C syncs from B, B syncs from A and they do it only once in an hour, could result in a huge time gap between commit on server A (archlinux.org) and sync on server D. In fact, I think this is also important because of our mirroring scheme, concerning that we don't sync from archlinux.org any more - so syncing could take 30 minutes until a new file gets from archlinux.org to an actual mirror from where users update.
As I am tired I may have written weird stuff, ignore it please.
To every mirror admin subscribed to this list: please voice your opinion on the issues raised in this mail. Since you are the guys that provide the service
- it is important for us to provide you with possibilities to make it better.
[1] https://www.archlinux.de/?page=MirrorStatus;orderby=delay;sort=asc
On Sat, Jun 19, 2010 at 23:41, Mihael Pranjić mpranj@limun.org wrote:
Am 2010-06-19 14:11, schrieb Roman Kyrylych:
Do you suggest to increase sync time to less than 1 hour? Perhaps we could try that.
Yip, I think syncing more often could improve the situation, since it does not really make much more traffic. It's rather a problem if you sync only once a day so the main mirror has to deal with more traffic in less time. Syncing every 15 minutes (on a random minute) could even improve performance since we would sync 100mb now, 100 later, instead of having a dozen servers syncing 200mb at the same time. I could also be wrong.
I see your point. Well, if there are no difficulties for mirrors with using more frequent updates, I see no reason why it should not be possible, especially after we completely turn off rsync access to archlinux.org for all non-tier1 mirrors soon.
However, another result of this (if applied to all mirrors) is that some probably don't sync from the tier-2 mirrors. Say server D syncs from C, C syncs from B, B syncs from A and they do it only once in an hour, could result in a huge time gap between commit on server A (archlinux.org) and sync on server D. In fact, I think this is also important because of our mirroring scheme, concerning that we don't sync from archlinux.org any more - so syncing could take 30 minutes until a new file gets from archlinux.org to an actual mirror from where users update.
Yes, there are some mirrors currently, that use a longer sync chain, but as more mirrors move to 2-tier scheme, I ask admins to sync only from tier1 mirrors (which all sync once per hour).
Mihael Pranjić (mpranj@limun.org) wrote on 19 June 2010 22:41:
Am 2010-06-19 14:11, schrieb Roman Kyrylych:
On Thu, Jun 3, 2010 at 20:31, Mihael Pranjić mpranj@limun.org wrote:
I think a bleeding-edge linux distribution should definitely not have issues with slow syncs. When there is a security update or an update like KDE it should be out there as fast as possible, no matter which mirror you use.
Do you suggest to increase sync time to less than 1 hour? Perhaps we could try that.
It'd be better that mirrors use push-syncing. At least the tier-1 ones should be required to do it to the downstream mirrors that ask for it.
participants (3)
-
Carlos Carvalho
-
Mihael Pranjić
-
Roman Kyrylych