[arch-dev-public] Arch mirror tracking woes

[Dan McGee]

On Fri, Feb 6, 2009 at 5:47 PM, Dusty Phillips <buchuki at gmail.com> wrote:
> 2009/1/21 Dan McGee <dpmcgee at gmail.com>:
>> On Wed, Jan 21, 2009 at 7:37 PM, Dan McGee <dpmcgee at gmail.com> wrote:
>>> On Wed, Jan 21, 2009 at 12:57 AM, Pierre Schmitz <pierre at archlinux.de> wrote:
>>>> Am Mittwoch 21 Januar 2009 03:32:54 schrieb Dan McGee:
>>>>> How does this look for a step in the right
>>>>> direction?
>>>>
>>>> Looks good to me. I use a similar approach for the server stats page. Do you
>>>> think it would be possible to store those data in the mirrorlist package
>>>> somehow?
>>>
>>> Can you point me to the page you are talking about?
>>>
>>> And what do you mean by storing it in the mirrorlist package? The
>>> intent here is to make the Django DB the canonical source of
>>> information that will be disseminated out to each of the places that
>>> need it- the downloads page, a mirrors page, and a way to generate the
>>> pacman mirrorlist.
>>
>> Some mostly-complete work on this stuff is on my branches:
>> http://code.toofishes.net/gitweb.cgi?p=archweb_dev.git;a=shortlog;h=refs/heads/working
>> http://code.toofishes.net/gitweb.cgi?p=archweb_pub.git;a=shortlog;h=refs/heads/working
>
> Hey all,
>
> Sorry for dropping the ball on this one. I've merged Dan's changes and
> am in the process of manually adding mirrors. I have entered all the
> details into the admin except the rsync IPs. I would appreciate it if
> someone would add those manually at some point. Then somebody can
> write scripts to automatically generate the rsync whitelist and pacman
> mirrorlist from this data. (Or did you do that already, Dan?)

I can try to do the rsync IPs this weekend, as well as making
scripts/etc. that spit out what we need. For the mirrorlist, we can
just make this a view on the public site- for the rsync IP list, we
might be able to make it a view on the dev site.

> In addition, I had to verify each of the mirrors to check if they have
> ISOs. In doing that I discovered some of the mirrors in the current
> database are broken. Specifically these ones:
>
> http://mirror.vmmatrix.net/ArchLinux/ (China)
> mirrors.jakimowicz.com (France)
> ftp://ftp.parrswood.net/Mirrors/ftp.archlinux.org/ (UK)
> ftp://saule.mintis.lt/pub/linux/ (Lithuania)
>
> They should be removed from the mirrorlist and rsync whitelist if they
> exist there, as I've already removed them from the DB.

Noted, and I'll try to remove them from the mirrorlist this weekend.

> I suspect there may be mirrors on the wiki page and/or in the pacman
> mirror list that are not in the database. It would be nice to get them
> all synced up.

Agreed. That was the point of this whole exercise.

> Finally, there are a bunch of unofficial mirrors listed here:
> http://wiki.archlinux.org/index.php/Mirrors#Unofficial_mirrors Is
> there a reason we shouldn't make some/all of these official?? Some of
> them may have been done already.

I think our only definition of "official" is "do they sync from
gerolde?". Not sure if that answers the question, but yeah.

-Dan