[arch-general] Searching reference server for mirror checking
Hello, i'm working on a script to check the sync status of the mirrors. Actually i use ftp.archlinux.org as a reference but Pierre Schmitz point to me that this is also only a mirror which must not be in sync. So is there a server (ftp or http) from where i could get the mtime, size and the repository db file as a reference? Pierre told me that he could run this script on his server, but for testing and bugfixing ;-) i need a reliable source. At the moment it seems i got too many false positives... Gerhard -- Never use a running system...
On 1/27/08, Gerhard Brauer <gerhard.brauer@web.de> wrote:
Hello,
i'm working on a script to check the sync status of the mirrors. Actually i use ftp.archlinux.org as a reference but Pierre Schmitz point to me that this is also only a mirror which must not be in sync.
So is there a server (ftp or http) from where i could get the mtime, size and the repository db file as a reference? Pierre told me that he could run this script on his server, but for testing and bugfixing ;-) i need a reliable source. At the moment it seems i got too many false positives...
no. there is not a publicly accessible location of the repo db file other than mirror servers. ftp.archlinux.org is the best candidate. the two servers (round robin dns for ftp.archlinux.org) synchronize approximately every hour.
Gruesse! * eliott <eliott@cactuswax.net> schrieb am [27.01.08 23:22]:
On 1/27/08, Gerhard Brauer <gerhard.brauer@web.de> wrote:
So is there a server (ftp or http) from where i could get the mtime, size and the repository db file as a reference?
no. there is not a publicly accessible location of the repo db file other than mirror servers. ftp.archlinux.org is the best candidate. the two servers (round robin dns for ftp.archlinux.org) synchronize approximately every hour.
Thank you. Then i must see if there are realy so many mirrors out of sync or if i have beasty little things in my script ;-) Maybe something with the UTC times on the mirrors... Gerhard -- Ich bin nicht die Signatur, ich putz' hier nur!
On 1/27/08, Gerhard Brauer <gerhard.brauer@web.de> wrote:
Gruesse! * eliott <eliott@cactuswax.net> schrieb am [27.01.08 23:22]:
On 1/27/08, Gerhard Brauer <gerhard.brauer@web.de> wrote:
So is there a server (ftp or http) from where i could get the mtime, size and the repository db file as a reference?
no. there is not a publicly accessible location of the repo db file other than mirror servers. ftp.archlinux.org is the best candidate. the two servers (round robin dns for ftp.archlinux.org) synchronize approximately every hour.
Thank you. Then i must see if there are realy so many mirrors out of sync or if i have beasty little things in my script ;-) Maybe something with the UTC times on the mirrors...
could be. When viewing with command line ftp i get.. ftp> modtime core.db.tar.gz core.db.tar.gz 01/27/2008 22:03:55 GMT apache shows me the following when viewing in firefox File: core.db.tar.gz 24 KB 27/01/08 10:03:00 PM So while the ftp says "GMT", it is really PST, as the times are very close, and it is clearly within the 10PM house where I am (PST timezone). Not sure why the time is being reported as GMT in ftp.
Hello! Ok, i'am 99% ready with the script (mostly output and "internally cleaning"). I put the output as txt attachement in this mail. Short: Overall Status -------------------------------------- Mirrors tested..................... : 49 Mirrors with connection problems... : 2 Mirror with data collection problems: 5 Mirrors OUT of sync................ : 25 Legend for txt file: size means *.db.tar.gz size reported by either FTP or HTTP mirror mtime also taken from protocols, converted to UTC if needed md5sum after doenlaod db file. Some mirrors seams to have the correct file size, but mtime and/or md5sum differs. I've checked some of them and there seams to bee slightly corrupted files on there. Next i will version pumping on the script (it's a ruby script) from now 0.1 to 1.0 ;-) and then we could see if this is something usefull for us. Gerhard -- Ein Registry-Cleaner ist fuer Win-User der Bulldozer des kleinen Mannes.
On 1/28/08, Gerhard Brauer <gerhard.brauer@web.de> wrote:
Hello!
Ok, i'am 99% ready with the script (mostly output and "internally cleaning"). I put the output as txt attachement in this mail.
Short: Overall Status -------------------------------------- Mirrors tested..................... : 49 Mirrors with connection problems... : 2 Mirror with data collection problems: 5
Mirrors OUT of sync................ : 25
Legend for txt file: size means *.db.tar.gz size reported by either FTP or HTTP mirror mtime also taken from protocols, converted to UTC if needed md5sum after doenlaod db file.
Some mirrors seams to have the correct file size, but mtime and/or md5sum differs. I've checked some of them and there seams to bee slightly corrupted files on there.
Next i will version pumping on the script (it's a ruby script) from now 0.1 to 1.0 ;-) and then we could see if this is something usefull for us.
I think it would be a very nice thing to have someone run periodically, and maybe send reports to the dev team...so that we can know which mirrors are not syncing properly. As of right now, we mostly have to rely on end user reports of problems (reactive instead of proactive).
Hello! * eliott <eliott@cactuswax.net> schrieb am [28.01.08 17:19]:
I think it would be a very nice thing to have someone run periodically, and maybe send reports to the dev team
I think we on archlinux.de could do this. I'll talk with Pierre next time about that. Another question , i assume that that you could say "something about mirroring": I also talked with Pierre about some more mirrors for germany. I have a list with some potential candidates. Our question is: what is the right way to get them as ArchLinux mirrors? We could contact their ftpmaster, but when they say: yes, we've waited for you ;-) whom must i/we contact on archlinux.org for rsync and so far? Is there a "FAQ" or some document anywhere? Thank's & a nice day Gerhard -- Neulich auf dem Maennerklo: Linke Reihe, bitte hinten anstellen, jeder nur ein Kreuz...
On 1/28/08, Gerhard Brauer <gerhard.brauer@web.de> wrote:
Hello! * eliott <eliott@cactuswax.net> schrieb am [28.01.08 17:19]:
I think it would be a very nice thing to have someone run periodically, and maybe send reports to the dev team
I think we on archlinux.de could do this. I'll talk with Pierre next time about that.
Another question , i assume that that you could say "something about mirroring": I also talked with Pierre about some more mirrors for germany. I have a list with some potential candidates. Our question is: what is the right way to get them as ArchLinux mirrors? We could contact their ftpmaster, but when they say: yes, we've waited for you ;-) whom must i/we contact on archlinux.org for rsync and so far?
The best course of action is probably to: 1) Contact the mirror admin, requesting mirroring. 2a) If they want to mirror from the arch main server: Open a bug ticket. This way the pacman packager, as well as the arch server admin, can both have a reference point for addition of the new mirror. Include relevant text, contact information, etc in the bug ticket. 2b) Alternately, some of the servers that already mirror have rsync access, and new mirrors could be "teir2" by syncing off of them. In such cases, make sure to qualify in the bug ticket such information, as well as any additional relevant text, contact information, etc.
Is there a "FAQ" or some document anywhere?
Not yet.
2008/1/28, eliott <eliott@cactuswax.net>:
On 1/28/08, Gerhard Brauer <gerhard.brauer@web.de> wrote:
Is there a "FAQ" or some document anywhere?
Not yet.
Probably you may find useful some instructions (e.g. fancy mirror sync script - or at least the recommended rsync options) from this page: http://wiki.archlinux.org/index.php/Local_Mirror Besides what Eliott said about filing a report on our bugtracker you should also add your new mirror to this page: http://wiki.archlinux.org/index.php/Mirrors It will be moved from Unofficial to Official part once added to pacman's mirrorlist and to Get Arch web page. -- Roman Kyrylych (Роман Кирилич)
participants (3)
-
eliott
-
Gerhard Brauer
-
Roman Kyrylych